fm77

Thank you very much for that super analysis done with the help of AI - I really enjoyed reading that. May I ask are you paying for that service? And if so, how much?

Anyhow, I downloaded your ZIP file and looked into the disassembly. It seems that the disassembler simply disassembled byte by byte not taking into account that TURBO.COM is both, code and data. Since the x86 instruction set is very tense, pretty much every byte sequence turns into legal instructions. Even the ASCII strings were disassembled. Look at address hex4864 in the file for example - it should be the string "Write block to file" but got disassembled. I wonder how AI managed that obscure file.

Turbo Pascal Version 6 source code is online and easy to find. It is not the original source code written by Anders Hejlsberg but a fantastic reverse engineering job. Turbo Pascal 7 added just a little bit to the compiler (like "break", "continue", "inhertied") and added an optimize step to the code generator (eliminate multible "les di,…" for constant pointers). But in order to study the compiler version 6 should do it.

I was quite late to the previous „Happy birthday Turbo Pascal" entry so I repost, @admin: I hope thats ok.

For all who wonder why Turbo Pascal was so fast here some insights: https://news.ycombinator.com/item?id=38477688#38486185

For all who wonder why Turbo Pascal was so fast here some insights:

50% is certainly due to the language Pascal itself. Niklaus Wirth designed the language in a way so it can be compiled in a single pass. In general the design of Pascal is in my opinion truly elegant and compared to other programming languages completely underrated. Wirth published a tiny version of his compiler written in Pascal itself in a 1976 book called "Algorithms + Data Structures = Programs".

In the late 70s Anders Hejlsberg took that version and translated it into assembly. He certainly must have changed the codegenerator since Wirth's version emitted bytecode for a tiny VM whereas Anders version produced machinecode, however if you take a closer look especially at the scanner and parser of Turbo Pascal and Wirth's version you can see that they are very similar. Back then Anders was not so much a language guy in my opinion but much more an assembly genius. And that resulted in the other 50% of why Turbo Pascal was so fast:

-) The entire compiler (scanner/parser/typechecker/codegenerator/ and later the linker) was written in assembly.

-) The state of the compiler was held as much as possible in cpu registers. If e.g. the parser needed a new token from the tokenstream, all registers were pushed to the stack and the scanner took over. After the scanner fetched the next token, registers where restored.

-) The choice of which register hold what was also very well thought through. Of course the cpu dictates that to a certain extent but still lots of elegant usage of the "si"/"di" register in combination of non repetitive lodsb/stosb instructions were done.

-) The entire "expression logic" (expression parsing / expression state / code generation for expressions) was kinda object oriented (yes, in assembly) with the "di" register hardwired as the "this" pointer. If the compiler needed to handle two expressions (left expression and right expression), then one was held in the "di" register and the other one in the "si" register. Since the "di" register was hardcoded, you will find lots of "xchg di,si" in the codebase before a "method" (a procedure with the "di" register as a "this" pointer) will be called.

-) Clearly the cpu registers were not enough in order to hold the entire state of the compiler so heavy use of global variables were made. Global variables have the advantage of not needing a register in order to access them (e.g. "inc word ptr [$1234]").

-) Parameter passing was done through registers and were possible stack frames were avoided (too expensive), meaning no local variables (still heavy usage of push/pop within a procedure, does this count as a local?)

-) Parameter passing done through registers allowed procedure chaining: instead of "call someOtherProc; retn" at the end of a procedure just "jmp someOtherProc" was used to a great extent.

-) Jump tables everywhere. In general the compiler was quite table driven.

-) Avoiding of strings as much as possible and if needed (parsing identifiers / using filenames) then avoiding to copy the strings around as much as possible, meaning all strings were held in global variables. The big exception here was of course the copying of the identifiers from the global variable into the symbol table.

-) Starting with Turbo Pascal 4.0, hash tables were used as symbol tables. Arena allocator for memory management.

I am sure I forgot a lot, I reverse engineered Turbo Pascal back in the late 90s. Most of the above applies to Turbo Pascal 7.0, but lots have not changed in earlier versions over time.

It is a shame that such a wonderful codebase is buried under the "closed source, proprietary software" label. It is clear that today nobody would write a compiler the way Turbo Pascal was written, not even in a high level language but the codebase has some many tricks, so many elegant solutions, that it is a real pity that this is not open source. Of course the codebase is on the web, just not the official one.

Thank you Anders Hejlsberg for such a wonderful piece of software.

I have read that quite often here and there and while the management at Borland certainly was a pain, I don’t think that for Anders Hejlsberg this was the main reason to leave.

I believe (and I have no prove) that during development of Delphi V1.0 Anders realized that his „baby", the source code of the compiler core written in 16-bit assembly became worthless. The new 32-bit compiler for Delphi V2.0 was written in C by Peter Sollich.

Anders new role at Borland was just to be an architect / manager / teamleader / whatever… He could have contributed to the new 32-bit compiler in a way he does today to TypeScript, but for whatever reason he didn’t wanted to. That, along with the Borland management and the nice $$$ signing bonus offered at Microsoft made him leave. Again, that's just my take.

Peter Sollich left Borland I think in 1998 in order to join Microsoft, to this day he is in the C#/.net team. https://youtu.be/LPcjSdob9AA

And I still wonder who has written Turbo Pascal for Mac in 1985/86…

Slight correction from an Turbo Pascal expert ;-)

Turbo Pascal did NOT have an 8-byte fixed point "currency" type suitable for monetary calculations. It did have an int64 type called "comp" though which was handled as a float by the FPU and hence not slower than the types "single" (f32), "double" (f64) or "extended" (f80).

IIRC, "currency" came with Delphi V2.0 (or even later), but then still it wasn't slower than other floats when you did heavy calculations with it as it was also handled by the FPU. Only reads and writes from and to such variables were expensive as there was always a scaling (multiplication by 1e4 and 1e-4) involved — internally it was that int64 "comp" type. (But here I might be wrong, I never really used "currency", I disassembled lots of Delphi binaries with lots of different data types as I wanted to know how the compiler worked. Today however my Delphi knowledge is quite fuzzy).

In Turbo Pascal :-) you have to filter for NaN or else you will end up with a Runtime Error.

  const NaN = $FF shl 23;

  var x, t: longint;
      f: single absolute x;

  begin 
    t := 0;
    x := 0;
    repeat
      inc(t, ord((x and NaN <> NaN) and (0<=f) and (f<=1)));
      inc(x);
    until x = 0;
    WriteLn('total: ', t);
  end.

cproc: a C11 compiler

  *) https://github.com/michaelforney/cproc
  *) programming language used for the compiler: C
  *) using QBE as a backend - https://c9x.me/compile/

chibicc: A Small C Compiler

  *) https://github.com/rui314/chibicc
  *) programming language used for the compiler: C
  *) emits assembly text (x86-64)

In standard Pascal, functions are closures but not first-class values (can be passed as arguments to other functions, but cannot be returned or assigned to variables)

Interesting - being an absolute non-expert in closures - but isn't it the other way around? That is, in Pascal, functions are first-class values but not closures?

Unless I am miss something here, but functions can be assigned to variables:

  function add(a,b:integer):integer; { far; in Turbo Pascal }
  begin
    add := a + b;
  end;

  type MyFunc = function (a,b:integer):integer;

  var f:MyFunc;
      x:integer;

  begin
    f := add; { here we assigning a function to a variable }
    x := f(3,4);
  end;

I know what you mean here, but that's not entirely true. In Turbo Pascal you had two ways to receive a pointer to a function:

1.) the way just shown, and here you are right, Turbo Pascal doesn't allow local functions, but that’s because how local functions in Turbo Pascal were implemented. Local functions have a hidden parameter (a 16-bit pointer to the caller's stackframe) pushed onto the stack, how do you handle this hidden parameter when you deal with a function pointer with a defined function signature? To avoid headaches I guess Hejlsberg simply choose to not allow local functions here.

2.) via a regular pointer - you simply store the function address in a pointer variable (or pass the function address to a pointer parameter), so:

  procedure MyProc;

  { a local function }
  function sub(a,b:integer):integer; { far; in TP }
  begin
    sub := a - b;
  end;

  type MyFuncHidden = function (a,b:integer; hidden:word):integer;

  var p:pointer;
      x:integer;

  begin
    p := @sub; { assigning a local function to a variable }
    x := MyFuncHidden(p)(7,8,0);
  end;

Look at the implementation of ForEach/FirstThat in the Turbo Vision library. Both ForEach and FirstThat accept local functions as parameter (and only local functions).

Welcome to HN Philippe!

That compiler served as the blueprint for Borland tools.

When I was disassembling Turbo Pascal more than 20 years ago, I saw the similarities between the two, especially the scanner and parser where pretty much in line with the source code of Niklaus Wirth [1]. Even some of the global variables were the same. Back then I figured that since the source code of Niklaus Wirth was originally in Pascal, Anders Hejlsberg was actually "the Pascal Compiler" for the z80 (and later x86) translating the Pascal source into assembly. For me it seemed that Anders had a very good hand for register allocation, I remember the disassembly of Turbo Pascal was full of tricks, simply fantastic and elegant!

Here is one version Niklaus Wirth Pascal Compiler: [1] https://www.cs.hs-rm.de/~weber/comp/pascals3.html

Very interesting!

BTW: I think I found a typo: "MATCH2("+=", PERCENTEQUAL);" — should be "%=" ??

Well, not really. Consider the following:

  type
      ItemPtr = ^Item;

      SomeOtherType = record
        a,b,c:integer;
      end;

      Item = record
        Data:string;
        Next:ItemPtr;
      end;

Stop fooling around. It's time to get hardcore about software. With Microsoft.

And no phone calls, please. If we like what we see, we'll call you — Microsoft

haha, I just found [1]: (very nice I might add)

  Program PowerPascal;
  {$X+}
  {
    Who:  Michael Warot
    When: November 12,1989 (my 26'th Birthday!)
    What: The beginnings of a language compiler,
          takes source from STDIN, and generates
          Assembler Source for STDOUT
  }

    pp002.zip Power Pascal v0.002 (c) 1993 by Mike Warot
    Its a very old OS2-oriented Pascal compiler. 
    Source: .pas (Borland Pascal)
    Output: .asm (32-bit => Masm 6.0 + Link386 = > lx .exe)
    Documentation: comments in English

Interesting (funny) side note: Niklaus Wirth did return the very same token named colon for ".." as well as for ":" in his Pascal compiler (page 24 and 25 in [1]). So back then you could have chosen whatever you prefer… :-)

[1] https://www.research-collection.ethz.ch/bitstream/handle/20....

From the Byte magazine • November 1987 • p97:

"Borland says version 4.0 will outperform previous versions in compilation speed and efficiency. I compared 3.0 and my preliminary version of 4.0 in compiling the CALC.PAS program provided with Turbo Pascal. On a 4.77 MHz IBM PC with 512K bytes of RAM and an 8087 co processor, version 3.0 took 15 seconds to compile the 1272 line program. Turbo Pascal version 4.0 took just 10 seconds to compile a slightly different 1273 line version of CALC.PAS."

…he started…? no. He started using hash tables within the compiler as of v4.0.

…he stated (in that interview) using hash tables within the compiler as of v2.0? yes! ;-)

What I mean here is that Anders probably remembers that wrong which is absolutely ok, thats now what… 35 years ago…

oh, and by the way…

Young Anders Hejlsberg at age 28 in an interview...

https://community.embarcadero.com/files/2008/11/tp01_868.jpg

https://community.embarcadero.com/files/2008/11/tp02_871.jpg

https://community.embarcadero.com/files/2008/11/tp03_874.jpg

https://community.embarcadero.com/files/2008/11/tp04_877.jpg

IIRC, hash tables came with Turbo Pascal v4.0 and not with v2.0 (4 years between v1.0 and v4.0). Interestingly I can't remember what the difference was between v1.0 and v2.0 but looking at the release dates I think there wasn't much…

  Borland Pascal v7.0             27th October 1992
  Turbo Pascal for Windows v1.5    8th June 1992
  Turbo Pascal for Windows v1.0   13th February 1991
  Turbo Pascal v6.0               23rd October 1990
  Turbo Pascal v5.5                2nd May 1989
  Turbo Pascal v5.0               24th August 1988
  Turbo Pascal v4.0               20th November 1987
  Turbo Pascal v3.0               17th September 1986
  Turbo Pascal v2.0               17th April 1984
  Turbo Pascal v1.0               20th November 1983

…in terms of software, yes, Turbo Pascal is - at least for me - one of the finest software ever created… 20 years ago I reverse engineered the command line compiler BPC.EXE, 80kB of code, 40kB data, absolutely mindblowing whats in there. A masterpiece of software which should be open sourced (never going to happen…) Anyway, thank you Mr. Anders Hejlsberg for Turbo Pascal!

Windows 11 Enterprise - 20 GB download

four different virtualization software options: VMWare, Hyper-V, VirtualBox, and Parallels.

This VM will expire on 01/09/22.

This evaluation virtual machine includes:

:) Window 11 Enterprise (evaluation)

:) Windows 10 SDK, version 2004 (10.0.19041.0)

:) Visual Studio 2019 (latest as of 10/09/21) with the UWP, .NET desktop, and Azure workflows enabled and also includes the Windows Template Studio extension

:) Visual Studio Code (latest as of 10/09/21)

:) Windows Subsystem for Linux enabled with Ubuntu installed

:) Developer mode enabled

:) Windows Terminal installed

So if I understand that correctly, then the TypeScript compiler source consists of ca. 250.000 lines of code (LoC). According to the video, the scanner has ca. 2.600 LoC (~1%) and the parser has ca. 9.000 LoC (less than 4%), so in total scanner and parser together need ca. 5% of the entire codebase… Ok.

Yet over the years I have collected ca. 30 books about compilers and a lot of them talk way more about scanners and parsers (up to 70%). You go figure…

Walter, I am sorry, completely off-topic… but I have to ask - and you don’t have to answer ;-) …is that you?

https://archive.org/details/byte-magazine-1988-09/page/n148/...

Best part, I still have that Commodore VIC-1213 "Machine Language Monitor" [1] along with my beloved VIC-20. I also had (and still have) a character-editor [2] and with that setup I wanted to write a computer game.

Problem was, I didn't know how to write a game, so with the Machine Language Monitor I disassembled Jeff Minter's Gridrunner [3] and learned really wonderful tricks.

I remember, in Gridrunner there was a main loop with 20 JSRs or so and each JSR handled a piece of the game, querying the joystick, updating the spaceship, producing the sound, etc... (kinda cooperative multitasking)

It was a wonderful journey to learn all this stuff from that game and then write my own one...

Jeff, if you ever read this, thank you for that wonderful Gridrunner, not only was playing it a lot of fun, I learned a lot from you!

[1] https://retro8bitshop.com/product/commodore-vic-1213-machine... [2] http://www.lust-auf-nostalgie.de/vc20/04_software/commodore_... [3] https://en.wikipedia.org/wiki/Gridrunner

Young Anders Hejlsberg at age 28 in an interview...

https://community.embarcadero.com/files/2008/11/tp01_868.jpg https://community.embarcadero.com/files/2008/11/tp02_871.jpg https://community.embarcadero.com/files/2008/11/tp03_874.jpg https://community.embarcadero.com/files/2008/11/tp04_877.jpg

Look at the pictures, gee... they were really young! My heroes...

Knuth: "I think future languages should be a combination of a programming language and a document language."

Wirth: "Ada is an uneconomical language. It throws too many things at you... C is becoming popular because UNIX is written in C... But I don't think it's a step forward."

The Worst VIC-20 Ever - Part 2: https://youtu.be/_rjjCkcXknE

The most "funny" disassembler I have ever seen in pascal/delphi is the one from cheat engine...

https://github.com/cheat-engine/cheat-engine/blob/master/Che...

That one unit has more than 16000 LOC, its basically one huge case statement where each opcode is handled separately... super simple but certainly not DRY (don't repeat yourself). But I guess its one of those cases where you should avoid writing DRY code.

In the same manner is the fpdebug-disassembler from the free pascal team, I found that disassembler one of the most elegant one.

https://github.com/alrieckert/lazarus/blob/master/components...

It has some minor errors in the Rex decoding though, if I remember correctly....

I don't know what a "white box" reimplementation is (white-box cryptography?) but I guess you meant a clean room implementation (https://en.wikipedia.org/wiki/Clean_room_design) which I believe this is not the case here (I am wild guessing though). For a clean room implementation you need at least two guys, one reverse engineer who writes down his findings and a guy who takes that information and reimplements the same functionality according to the information the reverse-engineer provides.

But you are right in that it uses the same algorithms, structures and control flow as the real Turbo Pascal and therefore should be bug-for-bug compatible but is written in Pascal instead of assembly.