I think any normal man who said he found only one woman attractive would be lying -- to his partner, certainly, and probably to himself.
There's nothing wrong with finding another woman attractive. How could there be? Beauty gives pleasure to everybody. Even that evangelical zealot, C.S.Lewis, acknowledged it in A Confession. Among the things he specifically found beautiful were, "Peacocks, honey, the Great Wall, Aldebaran, the shapes of horse and woman..."
But finding another woman attractive is quite different from cheating. :)