Does a business have to serve me at their drive-thru when I’m on a bike?

Yes! As long as you’re in the City of Madison, at least. City code states that “[b]icyclist use of sales and service windows shall not be prohibited.” Don’t expect every employee at a drive-thru to be aware of this, though.