You both are forgetting we're talking about air power of the 1930s and 1940s, the Germans had no radar, little electronics (radio and that was about it) none of the modern aids we take for granted. Thus flying though the mountains is dangerous even on a good day, and on a bad day it's down right sucidial. One of the most dangerous jobs the US air force did in WW2 was flying 'the hump' (the mountains between India and South Asia) to resupply our troops and allies in Asia.
Thus just flying in a mountain is treachorous.
Add to that there was no smart bombs and often you had to cluster bomb the hell out of an area to hit the target. Which means that even if you drop a 100 bombs it's a 'success' if 10% hit and remember we're talking hardened structures here so they can take hits.
This did'nt stop Swiss towns like Schaffhausen and Zurich being Bombed during the War (by the Allies, and they were allegedly accidents, but it shows that it can be done). Also both the Axis and the Allies repeatedly violated Swiss airspace- they fought back and shot down some, and interred many pilots, but the vast majority got away with it.
Add to that the cold, snow, blizzards, it would be almost impossible to bring artillairy, you'd be attacking the high ground and so on.
So like I said it would at best be a pyhric victory especially for such a small country and that didn't even want to be involved.
The Swiss strategy for dealing with Invasion was to abandon the Cities and Economic Centres and retreat to a stronghold in the Alps, the
Reduit, from where they would fight a war of Attrition for the remainder of the conflict.
In other words, they thought that in an all-out war Germany would annihilate them.
I agree that it would have been foolish and wasteful for Hitler to invade Switzerland during the war (and he planned to; make no mistake about that- he was just sidetracked by Russia and Normandy Landings), but it would be far from a Phyric victory because even the Swiss felt that they could'nt match German military might.
Go look up the history of people successfully invading Switizerland, you'll see that precious few have managed it.
Oh and don't underestimate the Swiss army, for over 600 years Swiss mercs were prized and renowned for thier fighting and the Swiss arguably had better riffles then the Germans for most of the war.
It would definitely have been a pyhric victory considering how insiginficant and nonthreatening the Swiss were.
Don't underetimate the German War Machine, which was easily far larger and more powerful than a fighting force that, to fight any invasion from Germany, stood at a measily 500,000 strong.
The Swiss had shot down German fighter planes and it's Press had openly criticised the Nazi regime. It's state of Neutrality left it a haven for Allied spies (as well as German ones, of course), and it had to repeatedly convince the Reich that it's economic concessions and appeasment made it not worth the trouble (again, Hitler planned to invade them anyway).
The Axis had a tremendous psychological advantage over the Swiss, given that they surrounded them on every side, had a tremendous run of military successes, had overall superior military forces and technology, and had a reputation for butchery towards it's conquered nations (like nearby Czechoslovakia). As the OP says, one Swiss officer said that Germany could have marched on the Capital with just one Tank regiment. Two of it's main "ethnic" groups were Germans (the majority) and Italians (though they had a strong sense of Swiss identity), and the Axis links would'nt have been unnoticed by everyone.
Were it not for more pressing matters Hiter would have invaded Switzerland, and he probably would have won. Also the people that managed to succesfully invade Switzerland included Napoleon, who did it relatively easily (though the Helvetic Republic the French set up was not exactly what you'd call efficient), and later Austria and Russia (to fight the French). This proved that a Modern army (and the Germans were, of course, far more modern than that) could take over Switzerland without too much trouble (keeping it under control is the trouble); it was also the last time Switzerland was involved in a war with a foreign power (directly).
So really, for all their military prowess the last War Switzerland fought ended in it being invaded-twice- and only because it was in the way of it's enemies warpath (Napoleon was on his way to Austria; actually; I'm not sure if it was Napoleon or just another French army).
Regardleess of whether or not it was foolish to attack Switzerland (it was not as foolish as you think), the question was why did Germany not do it.
You think it was because it would have been a military disaster- that is wrong. Hitler actually planned to invade during the war, and military strategies were drawn up to that end. The Nazi's did'nt act simply because they were distracted.
