Why are orbits curved

It is certainly possible to set up a satellite so that it has a circular orbit (a circle is just an ellipse whose foci coincide). Gravity can only pull in the direction toward the planet. The inertia of the satellite makes it want to travel in a straight line, but if it does so, its velocity is no longer perfectly perpendicular to the pull of gravity, so gravity pulls it in; this will remove part of the velocity, but as the satellite is also falling inward, it gets a new component of velocity due to the acceleration of gravity. In a circular orbit, we know that the ground speed is constant, so these two effects must perfectly cancel one another out to leave the speed of the satellite unchanged. Now imagine that we fire the satellite's boosters so that its ground speed increases. Now the desire of the satellite to go straight is stronger, so the two effects do not cancel perfectly, and the ground speed will vary. You can see how this corresponds to an elliptical orbit, and how a planet orbiting the Sun behaves in the same way. (Of course, planets have no boosters, but think about what effect the initial velocity of the planet due to the process of its formation would have--what happens if a planet is formed with only a small initial velocity, far from the Sun, or if it is formed with a large velocity, very near to the Sun? What happens if the inital velocity of the planet is zero?).

1) An orbit is by definition a circular path of an object around another object

2) A curve is created by two forces wich have effect in different direction: One being gravity pulling the object inside the circle, the other being the speed of the object pushing the object forward (as if in a straight line)

Only the perfect combination of the power of these two allows for the object to not get pulled in nor flow of and away.