Bringing Clang to Windows

http://blogs.msdn.com/b/vcblog/archive/2015/05/01/bringing-clang-to-windows.aspx

by intrepidsoldier • 11 years ago

142 66 11 years ago

blogs.msdn.com

twoodfin• 11 years ago

Where does C2 come in on the stack? Is it compiling/translating LLVM IR?

intrepidsoldier4• 11 years ago

The latter.

Phlow• 11 years ago

What's the situation with SEH these days in Clang?

spongo27• 11 years ago

http://clang.llvm.org/docs/MSVCCompatibility.html says minimal. Note that this is about cross-plat conformant c++ code. SEH is windows specific.

cokernel_hacker3• 11 years ago

That document is out of date. SEH works OK for 64-bit X86, 32-bit X86 is ongoing. Catching C++ exceptions has been implemented for 64-bit X86 but isn't ready for prime time.

revelation2• 11 years ago

I'm a bit confused about how this will work exactly. If I have Clang and VC++ compiled code trying to interop with each other, what happens if Clang decided to use a different memory layout for std::string (or any other type really) than VC++? What compiler is doing which task in the pipeline here exactly?

I mean, the reason everyone does interop strictly in C is that there are zero standards for how a compiler implements the C++ features exactly down-low.

ComputerGuru2• 11 years ago

As I understand it, it's not bringing Clang/LLVM's libc++ to Windows. The memory layout of, for example, std::string isn't decided by the compiler - it's decided by the library/header files. Both Clang and MSVC would be using the same MS C++ header files, so the only thing up to the compiler would anything that isn't specified in the header files, behind the scenes.

We're already using Clang with msbuild on Windows for native C11 support (come on, Microsoft!), courtesy of pre-built Clang for Windows binaries [1] - using the Microsoft C/C++ headers and linking against the Microsoft C/C++ runtimes.

There's already a plugin for MSVC that makes the Clang toolset available. VS 2013 (or maybe it was 2012?) decoupled the MSVC build chain/toolset from the IDE, making it possible to swap out which toolchain you're using from the project properties. Actually, it may have even been VS 2010, because I was using it to compile with ICC, but then again, I remember having a hell of a hard time so perhaps it was VS2012/2013.

1: http://llvm.org/docs/GettingStartedVS.html

spongo27• 11 years ago

correct, this will use MS libraries. If you watch Jim Radigan's Build talk (he owns C2), he talks a little bit about how the ABI is really a part of the platform and it might clarify some of this. http://channel9.msdn.com/Events/Build/2015/3-714 about 23 minutes in. Thanks, Steve (vc dev mgr)

Someone• 11 years ago

"The memory layout of, for example, std::string isn't decided by the compiler - it's decided by the library/header files."

Not quite true. Different compilers can decide to do different alignment/padding. It should be fairly simple to make clang and MSVC avoid that scenario, though.

Also, they must have taken care to use the same ABI and C++ name mangling rules in both compilers. I would guess Microsoft's version of clang follows MSVC here, so that one can link clang code to existing third party libraries.

cokernel_hacker3• 11 years ago

Clang has, for some time now, supported the MSVC ABI.

Here is their implementation of name mangling: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/AST/Micros...

Here is their implementation of record layout: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/AST/Record...

Here is their implementation of vtable generation: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/AST/VTable...

Here is their implementation of RTTI: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/Mi...

Here is their implementation of C++ throw metadata: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/Mi...

revelation2• 11 years ago

That's nice, I wasn't aware that they were working on this specifically.

bla2• 11 years ago

But none of that was written by Microsoft employees, so I'm a bit confused about what exactly this blog post is announcing. It feels a bit like they're taking credit for the work done by the clang community.

cokernel_hacker3• 11 years ago

To be clearer, "Here is their implementation of ..." should read as "Here is clang's implementation of ...".

AFAIK, Microsoft hasn't contributed to any of these ends. I, for one, hope that they do.

malkia2• 11 years ago

It's not easy to mix two different versions of code compiled with different microsoft compilers, or even with the same compiler but different settings.

mwcampbell2• 11 years ago

If the Windows team at MS thinks of c2 and the MS linker as part of the Windows ABI, then I guess the MinGW toolchain (and a hypothetical clang toolchain where LLVM generates the native code) will always be second class, and therefore best avoided by developers who want to target Windows, even for classic Win32 desktop apps.

coliveira2• 11 years ago

It all depends on how cutting edge you want to be in the windows platform. If you want access to the latest technology, MS tools will always be the necessary choice -- Borland learned this lesson after a lot of hard work. But if you want to create products using the stable version of the windows API, gcc is already a very good choice.

spiralpolitik• 11 years ago

I kind of disagree here. Compilers reached commodity status a long time ago. Nobody makes money off them compared to how it was in the 90s and the list of ex-compiler makers is as long as your arm (Watcom, Metrowerks etc).

'clang' is probably approaching the point that its good enough so the question for Microsoft is do they still see any value in continuing the develop their own compiler or should they just switch to 'clang' ?

I think this is the big shift we are seeing in MS this year. Instead trying to owning everything, they seem to be shifting their focus to owning the parts that have value and commoditizing the rest. The strategy was successful for Apple so its unsurprising that Microsoft is looking to a similar strategy as part of their salvation.

HelloNurse• 11 years ago

Latest technology? What latest technology?

If you mean C and C++ features, Microsoft standard conformance has slowly improved over the years from not giving a fuck to reasonable, but the only bleeding edge tools they've ever created are experimental research projects. They are completely out of clang and GCC's race to implement C and C++ standards and future standard changes.

If you mean some new API, they are libraries and all compilers have equal access to them.

davidgerard5• 11 years ago

and is fast becoming the preferred C++ compiler for non-Windows platforms.

They wish. Microsoft still hates GNU.

ComputerGuru2• 11 years ago

It is, though. And Microsoft isn't alone in hating GNU - FreeBSD no longer ships with or uses GCC; the entire FreeBSD kernel and userland is built with Clang these days. And, of course, Apple's the one that started this whole stampede away from GCC in the first place. LLVM/Clang really and truly have hit GCC hard - and definitely for the best. Look at the improvements to GCC spurred by newer features in the Clang build toolchain.

EDIT:

In reply to davidgerard's comment below:

I will not believe this falsifiable claim without numbers. FreeBSD has almost no users compared to Linux.

Non-Windows does not mean Linux.

And what falsifiable claim? They didn't say Clang is the most popular - they said it's becoming. But if that's really the game you want to play, then it's Linux that has almost no users compared to the number of iPhones/iPads/iWatches/MacBooks out there - Clang/LLVM-powered, each and every one of them.

davidgerard5• 11 years ago

I will not believe this falsifiable claim without numbers. FreeBSD has almost no users compared to Linux.

istvan__4• 11 years ago

There are several reasons why FreeBSD is ditching GCC, one of them is licensing. I am wondering what are the technical reasons though...

michel-slm• 11 years ago

GP actually used falsifiable properly - meaning that the claim is testable (evidence could swing both ways). A non-falsifiable claim would just have to be taken on faith alone or rejected altogether

istvan__4• 11 years ago

Not much of a hate but rather protecting their IPs by avoiding licenses like GNU. They are not alone in this, most of the big companies are avoiding it. Some companies use it and hoping for not getting caught (VMWare). I still don't see hate here, MS is just smart enough to avoid these sort of situations.

davidgerard5• 11 years ago

Services for Unix was full of GPL stuff. They're not protecting anything they have any sane threat model on. It's Microsoft being Microsoft.

yarrel2• 11 years ago

Microsoft is on record as hating the GPL.

If that's changed it's not showing in their careful use of the (very fine) Apache license rather than the GPL.

istvan__4• 11 years ago

Hell yeah! Microsoft just became one of the viable platforms this week in the cheap segment with reasonable licensing.

yarrel2• 11 years ago

Still not in the free as in speech and beer segment though.

malkia2• 11 years ago

Wow! Microsoft! Wow! You should've had Satya Nadella as CEO ages ago! Congrats!

phkahler• 11 years ago

I'm starting to see a strategy here. I've been wondering how Microsoft can make money from all.this open source activity. Their platform is becoming obsolete, but is being replaced by OSX, iOS, and android. That makes developers unhappy because of the multiple OSes they need to support. But MS can step in and bring their own platform as an abstraction over these others and make developers happy. Build once, target everything. I'm betting key pieces will not become open source, they want to have control over the new platform. That is the midterm goal - regain control (even a little bit) of a dominant platform.

iamcreasy• 11 years ago

(sigh)...

wvenable• 11 years ago

I disagree because bringing Objective-C, Java/Android, and CLang compilation to Visual Studio doesn't provide any sort of abstraction.

The support for Objective-C, Clang, etc is, in my opinion, merely about bringing iOS and Android software to Windows (especially for phones and tablets). Microsoft knows they need current high quality apps on their platform to get users. Users which will buy their devices and operating systems. They don't really want developers to run iOS software unmodified on Windows but they do want to make it really easy for them to attempt a port.

PeteJacobs23• 11 years ago

Windows is not becoming obsolete. Quite the opposite. In the enterprise Windows will stay for a long time. I'm working at company > 500000 employees and our IT will be very happy when they can replace the iOS devices with Windows equivalents. And certainly the Desktops aren't going to be replaced by OSX. Even more with Windows 10 around.

MS strategy is much more straight forward. Simply offer reasons and make it easier for customers to use their products. Mainly the cloud and mobile services. They're just smart enough to understand where Google and Facebook make their money with - not by selling OS.

Demiurge• 11 years ago

I agree. We've been using OpenLDAP for 3 years now, and it has been a very scary, black magic experience, just trying to get basic stuff like DNS, NFS, mac, windows clients, working. Hoping to move to Windows AD as soon as possible.

krylon• 11 years ago

Given the amount of application code that has been written for Win32 over the years and that is still in use, I do not see any viable alternative to Windows on corporate desktops.

Sure, you might have VDI and access a Windows VM remotely from your iPad or whatever, but Windows is not going away for a long time.

(And don't get me wrong, I would love replacing every single Windows desktop at our company with Debian or CentOS or even OSX, but unless a lot of companies decide to port their applications to one of these systems, it is not going to happen.)

DannyBee3• 11 years ago

I agree with folks who think this is a bit confusing.

This blog post is announcing:

1. Using clang as a frontend.

2. Using C2 (MS compiler) as a backend.

What the LLVM community (IE not MS so far, but participation is of course, welcome)) has been building:

1. Using clang as a frontend

2. Using llvm as a backend

and even more specifically

1. Using MS Headers + MS Libraries

2. Using clang as a frontend

3. Using llvm as a backend.

(see blog.llvm.org/2014/07/clangllvm-on-windows-update.html for a 9 month old view)

Some of the mingw32 folks have been working on replacing #1 with non-ms stuff, but let's put that aside for the sake of this description )

Using clang as a frontend is where almost all of the ABI compatibility happens anyway (and where various developers have been focusing), so you end up with VC++ ABI compatible binaries whether you use C2 or not, if you/they use the pre-existing clang stuff.

Whether they are doing that (and then lowering llvm ir to c2 ir) , or if they've built an alternative IR generation with clang (IE generating c2 ir directly from clang), that they haven't released, is not clear.

If they've built something that generates C2 IR from clang, they would have had to write their own ABI compat, which would mean it may be different than the binaries clang would build.

(exception handling is also codegen'd by llvm, so exception handling could end up different, though of course, compatibility is the goal)

clord• 11 years ago

We went through this at my former employer (not MS) just before i left. In fact the politics surrounding it drove me to seek relative sanity of doing a startup. When I started working on it, there really were two camps: clang emitting house IR instead of LLVM IR, or translating LLVM IR either early (right out of clang) or late (as a target after all optimization). I was working on the prototypes for the latter pair of options before another constraint was added that invalidated all of those paths.

The problem was that clang is quite intertwined with LLVM IR. ABI is spread all over the place, both FE and BE. It was proving hard to get something the house optimizers/code generators would be happy with. So a more radical path was chosen — wiring clang's parser + ASTs to another C++ front-end's AST and then using the old IR generator to get all of the idiosyncrasies right for the target. It was wacky, but seemed to be half-way working when I left, such that major open source code was compiling.

The whole idea of taking just parts of clang/llvm reeks of a half-measure anyway. It's probably better to either keep the existing compiler stack, or cut it right out and put the secret sauce into LLVM as add-in modules.

intrepidsoldier4• 11 years ago

if the other compiler doesn't happen to have a great AST story, then this approach looks difficult.

spongo27• 11 years ago

Hi DannyBee, sorry if we didn't get the message out clearly. Jim Radigan gives a deeper technical explanation in the build talk he did yesterday. http://channel9.msdn.com/events/Build/2015/3-610 It's translating LLVM IR to C2 IR. We know that the Clang/LLVM community is hard at work on implementing the MS-ABI and we've reached out to a few developers working on ABI stuff as well as symbols support. We intend to engage with that community more openly now that we've announced but of course shipping the code will be our top priority. One important contribution of the Clang/C2 approach is that we will have full native VS debugger support. This is just the first step in a longer journey. Thanks, Steve, VC Dev Mgr

BruceM• 11 years ago

Hello! Is there any chance that this would work with other language front-ends that generate LLVM IR?

mwcampbell2• 11 years ago

Can the clang+LLVM toolchain generate a PDB file that can be used by WinDbg? Can it generate .obj and .lib files that can be statically linked with those generated by Visual C++?

chandlerc1024• 11 years ago

No, it can't generate a PDB file itself, at least not today. Maybe in the future the MSVC linker will be able to generate one even though you compiled with Clang/LLVM (it does that already today but the PDB file is almost empty).

You can definitely build obj and lib files and link them with files built by MSVC. That works well.

We also support putting DWARF debug info (the standard format) into the binaries, and the LLVM linker LLD will link them and you can debug the resulting program with the LLVM debugger LLDB (or GDB for that matter).

__Joker• 11 years ago

The broader stroke is MS to be relevant in the platforms side of the story. Sometime people will argue that windows is used extensively in enterprises, but obviously the today startup which will be tomorrows enterprises are using almost no MS eco system.

nerraga• 11 years ago

At approx. 26min he mentions four processors that their code generator has to target: ARM32, x86, x64, and a new one coming out in the fall. What would that be? ARM64? *edit - never mind. Last five seconds they hint at it as arm64. MS is definitely doing some interesting stuff these days.

brentshields• 11 years ago

"So, any minor divergence in features of Clang and Visual C++ will cause you as a developer to have to worry about these compiler differences or limit you to using the smallest subset of the two compiler features"

In other words, the set of vc++ features.

intrepidsoldier4• 11 years ago

Yes. We know we are behind on conformance.

Bringing Clang to Windows

Related Stories