Great ideas in theoretical computer science

https://www.cs251.com/

by sebg • 7 months ago

203 46 7 months ago

cs251.com

Ifkaluva• 7 months ago

I seem to remember this specific class at the CMU School of Computer Science being described as a “weed-out class”.

MangoToupe4• 7 months ago

Theory is certainly a weed-out class. I think algorithms is certainly more difficult for a dedicated student tho.

wrs• 7 months ago

When I took the theory of computation class at CMU in the mid-80s it was in the philosophy department. The professor knew almost nothing about actual computers. Which was pretty cool, honestly.

arethuza• 7 months ago

"Computer science is no more about computers than astronomy is about telescopes."

Usually attributed to Dijkstra.

hearsathought• 7 months ago

When I took the theory of computation class at CMU in the mid-80s it was in the philosophy department.

To be fair, theory of computation is more logic/proofs/formalism than what is mistakenly associated with CS nowadays - programming. Besides isn't everything just philosophy at the end of that day?

The professor knew almost nothing about actual computers.

The foundations of theory of computation were laid before actual computers.

Which was pretty cool, honestly.

I can't imagine my philosophy professors teaching theory of computation as they too didn't know anything about computers. But I'm sure they would have made it interesting.

sota_pop• 7 months ago

I always thought it should be called “ComputING Science” as opposed to “ComputER Science”.

awefpojoi• 7 months ago

It is indeed a weed-out class for the CS majors. It's fairly difficult the whole way through, and the difficulty jump afterwards for required classes is much more manageable. I never struggled with a required CS class after I managed to get an A in this one.

jtbetz22• 7 months ago

I am old enough to remember when it was 15-199, taught by Steven Rudich, titled "How to think like a computer scientist".

They had to run it for a few years before they realized CS kids who did poorly in the class dropped the major - the implicit signal being "you don't know how to think like a computer scientist".

MangoToupe4• 7 months ago

I notice "highlights" is essentially empty, which seems to be the referent of the title of this post

Jtsummers• 7 months ago

The title of the submission is the title of the page, it's not referring to just the one section but the entire course.

MangoToupe4• 7 months ago

I see. Where can we see these great ideas?

Neywiny4• 7 months ago

This is computer science. My uni's course number wasn't too different and I remember 3 things worth sharing here: 1. Somebody asked the lecturer what practical application something had. He pondered for a bit, and said "I don't really care." And then gave an explanation of how it's a science/theory class. 2. A classmate threw fits in the group chat about how he'd never have to do this kind of work after graduating because he could hire people like our lecturer to do it for him. 3. The rush when I figured out how to prove something during the last problem of an exam. As the time ticked away and I'm just staring at the words over and over, before I can sink an ice pick in and finally start grabbing a foothold.

Other things not really worth mentioning were that we had some useless digital logic section at the start where we made a full adder and called it a computer. As a CompE, it was weird. The CS students thought they knew all there was to how a computer worked from that. Also, he was only a lecturer because our processor got sick right before the class and they found a grad student to do it. He was ok but took shortcuts and our TA would be like "oh, he did this fast and loose, so now I have to teach you the real way it's done".

It was a great class in retrospect and certainly pushed my boundary on theoretical computing but you could feel the code monkeys regretting their decisions each homework and exam. It's what radicalized me to believing we needed programming and computer science to be different majors.

AdityaSanthosh• 7 months ago

Hi, I am one of those code monkey you mentioned. I never took affinity to computer science/math, but I love building real world products with software. I built some really hard and interesting products. Basically, I love playing with tools and building tools from programming paradigms I know. Right now, I am struggling with all the interviews that have these leetcode problems. What sort of career in IT do you think would be best fit for me?

Neywiny4• 7 months ago

Well what I'll say is this: my job never had leetcode. Embedded engineering, especially if you do FPGA work, is very different from what leetcode has. Honestly if recruiters are using it for jobs like mine, they really don't get it. But I don't know you nearly enough to know. There are so many different fields up and down the stack. Front end, backend, embedded, cloud, edge, consumer, IoT... The list goes on. I would cast a wider net, I guess.

whateverboat• 7 months ago

There are three parts of building products in software just like everywhere else:

1. understanding what product needs to be built

2. Having the technical expertise to build them

3. how much effort is needed to build it

And all three things can be hard in themselves. The product can technically be just straightforward (even if somewhat tedious) but you should know what to build because you should know what the customers need.

The product can be technically so challenging that without the theoretical background, you would not even know that such a product would be possible. (An example here would be something say that requires distributed and independent time clocks).

Finally the product can be something that is technically simple, obvious in its customer demand but requires quite a bit of effort and therefore requires you to be good at procuring resources and managing them.

You need to figure out which of these three things made you successfull with the product you call really hard and interesting and pursue that line of industry (even if it is not software!). And then slowly try to become good at other two things.

lelanthran3• 7 months ago

Right now, I am struggling with all the interviews that have these leetcode problems.

There's no need to struggle with them. You'll cover fully half of them simply with experience with trees. Build them. Traverse them top-down. Then bottom-up. Search it, Balance it, then rebalance on modification. Invert it, prune it, Roundtrip it to JSON, etc.

After mastering that, given a problem of "develop a flood-fill algorithm to this specification" should be a walk in the park.

czhu12• 7 months ago

I was one of those students initially but then came to appreciate the beauty in the science and nature of computation. I always tell people that computer science is not about programming -- its about studying the dynamics of problems in nature.

Unfortunately, I realized this a little late, and it wasn't until towards the end of my final year when I started appreciating these courses. I long for a day I can return to studying these topics.

skipants• 7 months ago

radicalized me to believing we needed programming and computer science to be different majors.

Some universities offer Software Engineering as a BEng as well as CompSci as a BSc. At least in Canada.

Neywiny4• 7 months ago

But software engineering as a concept still isn't writing code. I'm a bit of a stickler about it as somebody who has an engineering degree, but when programmers with a CS degree say they're a software engineer, they're not. Software engineering as far as I understand it from the little bit I did in school is actually engineering. Requirements analysis, breaking down the problem, following methodology, etc. It's not just that they're writing somewhere.

So really there should be 3 fields of study: 1. The theory - computer science 2. How to apply the theory - software engineering 3. How to turn those designs into reality - programmers

It's like the mech engineering side. You have materials science and stuff, then mechanical engineers, then machinists.

suggeststrongid• 7 months ago

Is code monkey derogatory here?

Neywiny4• 7 months ago

No. As somebody who lives in the trenches, I'm not too far behind. I've met people who say they can only do theory and can't do the practical side of this area. I think they might use it derogatorily but anybody who does clearly doesn't understand that both sides are necessary. So if anybody says it in that tone, just walk away.

planet36• 7 months ago

How about "Software Simian"?

https://ia601605.us.archive.org/view_archive.php?archive=/23...

mallic• 7 months ago

we needed programming and computer science to be different majors.

Neither are needed anymore if people will be able to tell a chatbot what to do.

Instead you’ll need a Problem Solution Verification minor.

skulk• 7 months ago

Randomized algorithms are so damn cool. They really feel like cheating your way out of NP problems. I highly recommend that anyone interested in algorithms studies them.

another_twist• 7 months ago

+1. My favourite bit is when pivots are chosen randomly in quicksort, we get linearithmic expected complexity. The CLRS proof using indicator random vars was a oh-shit moment.

gnull2• 7 months ago

Btw, you can make quicksort deterministically O(n log n) if you pick the pivot point with linear median search algorithm. It's impressive how randomness lets you pick a balanced pivot, but even more impressive that you could do the same without randomness.

tmoertel• 7 months ago

Similarly, quickselect with random pivots runs in expected linear time.

For a proof of this, see https://github.com/tmoertel/practice/blob/master/dailycoding...

gnull2• 7 months ago

https://www.wisdom.weizmann.ac.il/~oded/PDF/rnd.pdf

Then you reach derandomization and it hits you once again, it shows you that maybe you can "cheat" in the same way without randomness. Not for free, you usually trade randomness for a bit more running time, but your algorithms stay deterministic. Some believe all probabilistic algorithms can be derandomized (BPP = P), which would be a huge miracle if true.

senthil_rajasek• 7 months ago

Earlier discussions:

March 15, 2024 Great ideas in theoretical computer science

https://news.ycombinator.com/item?id=39720388

93 comments

gorgoiler• 7 months ago

Module 8 is worded in a way that surprised me: “if we could decide the languages in NP efficiently, this would lead to amazing applications.”

My understanding of P=?NP is that all intuition points to the answer being “no”. The openness of the question doesn’t give us hope that a magical algorithm might one day arrive. It just means that despite a lot of suspicion that NP cannot be solved in polynomial time, we cannot yet prove this to be the case.

I suspect the answer to BANKACCOUNT=?1_000_000 is “no”, but although my card stops working every month, it starts working again on the last Friday of the month! My research team and I hope to visit an ATM one day to prove my bank balance is meager but until then it remains an open question (albeit likely false) in Gorgoiler Science.

IsTom• 7 months ago

There were surprising results in computational complexity before. We can't prove that P /= PSPACE which should be much easier to prove and yet people are stuck on trying to prove P /= NP. Personally I don't think that P = NP and if it were it would be very surprising, but "very surprising" in not "out of the question".

aleph_minus_one• 7 months ago

Module 8 is worded in a way that surprised me: “if we could decide the languages in NP efficiently, this would lead to amazing applications.”
My understanding of P=?NP is that all intuition points to the answer being “no”. The openness of the question doesn’t give us hope that a magical algorithm might one day arrive. It just means that despite a lot of suspicion that NP cannot be solved in polynomial time, we cannot yet prove this to be the case.

I admit that I am slightly biased towards that P=NP does indeed hold. I know that this is a non-mainstream view; even though I could give arguments to actual experts in the field that (surprisingly) did convince them that it is much less "clear" than they thought all the time that P \neq NP "must" hold.

On the other hand, I consider it to be plausible that the fastest algorithm that might exist for a "natural" NP-complete problem (e.g. 3-SAT) that exists could have a runtime of \Theta(n^(2^(2^(2^1000000)))).

Or, it is also possible that the proof of P=NP might be highly non-constructive, and obtaining an actual algorithm from this non-constructive proof might be even harder than proving P=NP. Such a situation actually exists in the Robertson-Seymour theorem ("every family of graphs that is closed under taking minors can be defined by a finite set of forbidden minors"):

https://en.wikipedia.org/w/index.php?title=Robertson%E2%80%9...

Unluckily,

https://en.wikipedia.org/w/index.php?title=Robertson%E2%80%9...

"The Robertson–Seymour theorem has an important consequence in computational complexity, due to the proof by Robertson and Seymour that, for each fixed graph H, there is a polynomial time algorithm for testing whether a graph has H as a minor. [...] As a result, for every minor-closed family F, there is polynomial time algorithm for testing whether a graph belongs to F: check whether the given graph contains H for each forbidden minor H in the obstruction set of F.

However, this method requires a specific finite obstruction set to work, and the theorem does not provide one. The theorem proves that such a finite obstruction set exists, and therefore the problem is polynomial because of the above algorithm. However, the algorithm can be used in practice only if such a finite obstruction set is provided. As a result, the theorem proves that the problem can be solved in polynomial time, but does not provide a concrete polynomial-time algorithm for solving it."

So, it is in my opinion a very "religious" belief that even if we could prove P=NP, a practically relevant algorithm will arise from this proof.

saghm• 7 months ago

It's worth keeping in mind that regardless of whether P = NP, we might never prove it one way or another. We already know from Godel that some statements can't be proven true or false in a given logical system (without introducing inconsistencies), so it's certainly possible that this is one of them. At the risk of stating the obvious, this quite different from the hypothetical you give where we definitively know that it's provable.

BrokenCogs• 7 months ago

There used to be a 'sophomore college' course at stanford called "Great Ideas in Computer Science" which I completed a while ago. I have great memories from that! https://stanforddaily.com/2015/11/02/classy-classes-cs-54n-g...

rivetfasten• 7 months ago

Who reaches for a tool they forgot about?

Few organizations invest in solutions only understood by one or two individuals on their team. This is actually what prevents undergraduate cs knowledge from having an impact. It makes me sad.

Undergraduate CS isn't about the things you do most of the time. It's about enabling the occasions where the alternative is to give up and shrug, or perhaps speculate instead of evaluate.

Undergrads, read this before taking a theory of computation class: https://swtch.com/~rsc/regexp/regexp1.html

turingbook• 7 months ago

Discussion 2 years ago: https://news.ycombinator.com/item?id=39720388

The similiar course in

- Stanford: https://web.stanford.edu/class/cs208e/cgi-bin/main.cgi/

- MIT https://ocw.mit.edu/courses/6-080-great-ideas-in-theoretical...

- Harvard https://beta.my.harvard.edu/course/COMPSCI1/2026-Spring/001

Great ideas in theoretical computer science

Related Stories