Why do electrons lose potential energy when shifting from a less electronegative atom to one that is more?

I thought that it would be opposite since it takes more energy to rip an electron from an atom that is more electronegative, therefore doesn't that mean it has more potential energy? Right now my book is explaining this in terms of redox reactions in covalent bonds. Can anyone explain this? Thanks!