Yes! As long as you’re in the City of Madison, at least. City code states that “[b]icyclist use of sales and service windows shall not be prohibited.” Don’t expect every employee at a drive-thru to be aware of this, though.
Madison Bikes... where anyone can ride a bicycle conveniently and comfortably to any place year round.