Physicist: About the same thing that happens to a straw in a glass of water: the water level in the straw evens out with the water level outside.
A tube from the ground to space would fill with air of about the same density and pressure as the air around the straw, decreasing as you go up until eventually you have a straw full of nothing surrounded by also nothing (in space).
What holds the atmosphere to the planet is gravity, so if a patch of air tries to drift off into space it literally falls back. A straw alone wouldn’t change that. On the other hand, if you attached some kind of pump to the bottom of the straw to make it have a higher pressure than sea-level, then you could pump air up the straw and have some kind of massive space-fountain of air (the air coming out would fall back to Earth just like water in an ordinary fountain). In fact! There is a situation very close to that happening on Saturn’s moon, Enceladus.
Whenever air or water or whatever travels up a straw it’s being pushed by pressure from the bottom (there’s no such thing as sucking), and one atmosphere of pressure can only push so far. For something like liquid mercury that’s about 76cm, which is why the “1 atmosphere” of pressure is often expressed as “760mm of Mercury”. If a closed tube is taller than that, then the pressure (here on Earth) isn’t great enough to push the mercury to the top which leaves nothing at the top.
Same idea with air. If you have a long tube full of air with the top open to space and the bottom pressurized to one atmosphere (or 760mm Hg), then the column of air in the tube will be as tall as the atmosphere.
A straw doesn’t provide an “escape route”; our air is free to try to leave whenever. The atmosphere stays where it is because it’s made of mass and the Earth has gravity. It’s a little sobering to realize that there’s nothing between you and a profound nothing (space) but a thin layer of air held down by its own unimpressive weight.
The barometer picture is from here.