Which formula should be used to correctly calculate the monthly mortgage payment?

You could use the compound interest formula?

New amount = old amount x multiplier^number of years

However you'd have to convert the months into years - for example 3 months is 0.4 years

In the standardized calculations used in the United States, c is given by the formula: For example, for a home loan of $200,000 with a fixed yearly interest rate of 6.5% for 30 years, the principal is, the monthly interest rate is, the number of monthly payments is, the fixed monthly payment equals $1,264.14.