Yes — North Warwickshire does allow in-person visits, but the published information here only points you to book an appointment rather than confirming walk-in access. The council’s appointment page is here: book an appointment.
If you’re trying to visit the Council House in person, it’s safest to arrange a slot first, since that’s the only clear guidance shown in the available council information.