how is algernon's insistence that his "duty as a gentleman has never interfered with [his] pleasures in the smallest degree" ambiguous?

The statement by Algernon that his "duty as a gentleman has never interfered with his pleasures in the smallest degree" could actually mean that while he is truly a gentleman, his pleasure and duty are the same thing. The audience however knows Algernon very well and clearly sees that he makes use of deceit in order to look like a real gentleman, while still living a life full of pleasure.