If I delete the emoji free app will I still have the emojis on my phone?

No because the you will have to enable the app on the keyboard and when you delete the app the keyboard will automatically disable the apps accessibility to the keyboard

If you delete the Emoji application from your phone there is a possibility the Emoji's will still hang around. It all depends on the way you set up the app on your phone. Some people opt to have settings and changes to the app saved on their SD card and some don't. If you save updates and changes to the app to your SD card you will still have remnants on your phone after deleting the app