Unit tests can ONLY verify if the functions return properly.
In theory, (nearly) all programming languages are Turing complete, so we may as well write... you know the argument. I would submit that unit tests can do far more than just verify input/output. To quote the XP school of thought, writing unit tests prior to code often results in better, more general, code. It certainly helps you to get past any design related "writers block". By using an "automated" unit test environment, I also have a decent form of regression testing (not to mentioned acceptance). Being a bit clever, unit tests can verify side-effects... is your scheduler really fair? Define "fair" in a unit test and find out. You can also verify that an algorithm is efficient and finishes execution in a certain amount of time (assuming you can figure out the worst case scenario).
Pragmatically, unit tests are extremely cheap compared to "test plans" and catch the vast majority of errors. It is actually pretty hard to get bugs in the interfaces between modules. Generally, if you can't effectively unit test a module, it is because the module is a tangled bunch of noodles. Refactor it.
Compare that to sensors in cars. You could have an oxygen sensor reporting the wrong O2 levels, but since it was reporting the correct type of value in the correct "range" (passing the 'unit test') you don't see a check engine light for the O2 sensor failure.
It is certainly going to be hard to unit test hardware sensors
with a software unit testing framework; However, I would bet that companies like Intel find a lot more hardware errors in their sims than they do by physically testing their hardware. I know this is true for satellites at Boeing (ie the virtual hardware sims vs the physical hardware tests).
To say that unit tests are the only worth while testing is really short sighted. Just because it is a long test plan doesn't mean it is correct, tests the right conditions in the right ways. No SINGLE form of testing will find 100% of bugs.
I don't believe that is what I said, but I do tend to agree with the sentiment. =) No form of testing will find 100% of bugs in non-trivial systems. NASA does testing that extends far beyond what any company would ever consider performing, and they still have catastrophic errors.
Are you certain of their qualifications? It just sounds like you described a bunch of managers. FWIW, they don't need to know the technology, although they SHOULD... a managers job isn't to design the system<snip>
I know that three were software developers and two of them were managers. I'm not sure about one of the team members. I believe that one of the managers had been a software engineer. The program was a horrible mess. In fact, it should have never been funded (long story... you can read McCain's comments on FCS if you're interested).
What makes this interview kind of funny is that the job req looked like a Microsoft marketing brochure... .NET this, SQL server that... There must have been eight or nine MS products listed in the req. During the interview, they asked me questions like "do you know linux", "do you know MySQL", "do you know C and Perl". I asked them about the obvious contradiction between the questions they were asking me and the job req. The manager replied that he should probably "update" the req. I felt like being a smart-ass and asking why they had switched from MS to Linux. He had obviously simply copied some other job req and pasted it in the form.
he just has to recognize talent and then put the right people in the right place.
Well, they aren't very good at that either... =)
I am willing to put money on it that my Dad (not in CS at all, but an Engineer, and a damn good one at that) could manage a group of people responsible for that, be able to hire well qualified people, and get the project going in the right direction without knowing what I know about CS, languages, and systems, etc.
Do you really believe that your dad could lead a team to create a compiler, operating system or database? Short of hiring Ullman or Tanenbaum to actually lead the project, no, I don't think so. However, your father shouldn't feel bad because I don't think that either of us could lead a team to create a nuclear reactor or a rocket propulsion system either. I'd be curious to ask what your going to ask someone regarding nuclear reactor design. Aside, from doing the obvious thing and hiring the Michael Jordan of the field, you really don't have too many options.
History is also tangentially on my side. The early Apollo missions were staffed and managed by aerospace personnel, and it was believed that it would be easier to train aerospace engineers to write software than other way around. The result was a smashing failure. If software engineering has taught us anything, it is that software is extremely hard to get right.
Software can be very subtle and misleading. No human could possibly beat a decent Othello program, but no Go program performs worth a snit. It doesn't seem rational to people not familiar with computational issues. If someone tells your father to create a billion watt stereo amplifier, despite being theoretically possible, it is easy to rattle off practical reasons for rejecting such a proposal. It is obvious. However, if someone suggests that you develop a program that determines whether another program "halts", it is pretty easy to convince yourself of the benefits of such a program ("We'll make MILLIONS!") and possibly make a snap decision heading straight down the road to theoretical impossibility. It's happened a thousand times on a variety of problems to the best of the best in the field. How could we expect someone outside of the field to know any better?!
* Assuming NP != P