The prayer book used on Rosh Hashanah and Yom Kippur specifically. The High Holiday services differ substantially from the rest of the year, so a regular siddur will not carry them — a machzor is the book people mean when they ask what to bring to shul.
The picture books on this page do it through story rather than instruction — a child follows a character through the apples, the honey and the shofar and absorbs the shape of the day. For under-fives, board books survive the handling better than paper pages.
Size and print. A machzor gets held for hours, so weight matters more than it seems in a photo, and large-print editions are worth it for anyone straining. If the Hebrew is not fluent, choose an edition with translation alongside rather than transliteration only.