The problem solves itself.
No matter if the reason is cultural or genetic; those who embrace hedonism disappear by themselves.
The hedonists no longer need to have to take the others into account to satisfy their objectives.
With all the means at their disposal, their move into their palaces were their robots do everything for them: they take care of the needs of the place and the owner, they provide for their distraction and pleasure.
Effectively, that people no longer belongs to the society, they have shut themselves out of it.
The people who are interested in other things beyond pleasure, continue to interact with other peoples. Those are the society.
It even allows you to explain why they are few hedonists, if any, in your current society: as technology made it easier, hedonists tended to isolate themselves, making it more difficult for them to reproduce. This gradually led to a decline in the proportion of hedonists, because either if it is due to genetical or cultural issues, the following generations would mostly descend from non-hedonists.
