HN user

b0b_d0e

368 karma
Posts3
Comments37
View on HN

South of Atlanta we had a tornado warning around 5am. Several downed trees and power outages but I haven't heard of any actual tornadoes that touched down in my area yet.

It seems unlikely that it'll ever get fully upstreamed, but according to a post on the discord that I'm pasting in full below, there certainly can be parts that may get upstreamed. I do a lot of NES related development in my free time, and llvm-mos has been awesome for rapid development. I'd love to see of this get upstreamed in the hopes that it could reduce the maintenance burden on the small team, but I'm not trying to speak for them or anything haha.

So, I wanted to do a little blurb on the topic of upstreaming LLVM. My previous answer to this questions was "yeah, we'd like to, but we have more work to do." This implied that we were working on it. More accurate to reality is that we were keeping it in the back of our heads and doing work to decrease the diff from upstream. The latter is also useful for making merges from upstream easier, and that's closer to the real reason I was doing it.

Well, I've lost some rather high-profile fights upstream. In particular, upstream now strips out attribute((leaf)) from LTO builds, which is the whole thing that makes our static stack solution work. I personally think this decision was totally bogus, and wasn't alone in this, but the conservative voice won out. My experiences with the LLVM community so far has been one of deep conservativism; the stranger you are, the more you need to justify your strangeness with real-world impact. We're a very strange hobby project, which just doesn't bode well. We could make our backend a lot less strange by making it a lot less good, but then it becomes impossible to compete with incumbents like KickC and cc65.

Accordingly, I'm not keeping the goal of maintaining llvm-mos entirely upstream in the back of my head anymore. I don't oppose work along those lines, unless it interferes with making llvm-mos the best 6502 compiler we can.

That being said, LLVM may independently change to be more amenable to us, so this may become easier in the future. This has already happened prior to us, with GlobalISel and AVR making development of the backend far simpler. If that happens, I'll definitely reexamine my opinion on this.

Alternatively, I'd definitely be open to upstreaming the unobjectionable parts of llvm-mos backend; we could then maintain the actual distribution as an increasingly thin fork from upstream. In fact, we could probably get started on that project today; I haven't yet spent much time considering the idea, but I'm starting to like it more and more, since it gives increased visibility, easier merges, and an excellent reference backend for upstream documentation. (We're really nice once you strip away all the crazy!)

The fact that its performant is tangential to the fact that its running on ARM android. IIRC Drastic still does a traditional JIT to recompile the guest ARM code into ARMv7a for phones (I might be misremembering, but I recall they had a whole lotta headache trying to update to support 64 bit ARM in time for the Android deadline for that. Also I remember them talking about tons of headaches dealing with the new storage APIs for Android)

Its really fast because of the tons of optimization that went into the application. It does many cool things to cut out overhead on the CPU side, but also it does a lot on the graphics emulation side, including hand rolled ARM NEON code for SIMD processing polygons. I'm not very familiar with DS emulation, but one of the devs for Drastic contributed to another project I worked on, so I had some chats here and there about these sort of things.

Emulation involves a lot of large byte arrays and static structs that your code updates a lot to represent the hardware

Things work a bit differently for "modern" emulators, where the emulators recreate the kernel/OS at a high level. In these emulators, the games will call into the system, and the kernel will be expected to do all thats necessary for the call. In the high level approach, this means that if a call allocates, so does the emulator (edit: note that this is a simplified view, as both emulators map a 4GB page that they work in for the guest system memory, but theres still a ton of side allocations that happen "outside" of the guest kernel). There is a lot of work that goes on in this layer of emulation, and theres going to be objects that the emulator allocates and later destroys. Process tables, thread lists, scheduler information, timing events, kernel synchronization primitives like mutexs, and so on to name some. I'm not intimately familiar with Ryujinx to make any statements about how they handle GC of course, but its something that they'll need to take into consideration. That said, there's plenty of other things like JIT compilation, shader compilation, caches filling up, and on and on that all also cause micro stuttering, so its not uncommon for even C or C++ emulators to have annoying pauses too.

fail0verflow did this exact thing when they first got linux working on the switch actually! https://twitter.com/fail0verflow/status/988543541403160576 It'd be a pain and a half to get a "proper" fast port with CPU emulation (either some lightweight JITing or natively running the code with hooks on SVC and such) and a GPU backend for the switch graphics. Not nearly simple enough to attract someone to do it on a whim with no other reason that to say it happened haha

The other sibling comments were talking about general C++ vs C# performance, but I wanna get into some emulation specific context here that's missing. I was one of the contributors to the yuzu project since it was first started, and so I'm fairly familiar with the challenges of switch emulation, albeit I'm not going to pretend like I was one of the all-star devs responsible for making the magic happen. (I've recently stopped contributing to the project in order to pursue other interests, so i'm going to keep the information here general and not specific in case somethings changed in the last month.)

Let's start off by breaking the core performance portions of emulation into a few broad categories. There's CPU emulation, for running the actual guest exe, Kernel and OS emulation for handling the system calls that games make, and GPU emulation for translating the guest's GPU work into modern graphics API that your PC can use. Now let's compare how language overhead will affect each of these main scopes.

CPU Emulation - Both yuzu and Ryujinx use JIT compilation to recompile the guest ARMv8 instructions into x64 at runtime. The specifics of the two emulators JITs are pretty different, and it'd be cool to go into more details, but the mile high view is a comparison of C# vs C++ isn't going to have much of an effect on the runtime difference. At least not near as much performance gap between techniques and optimization levels that the JIT is capable of. The goal of JIT compilation for CPU code is to remove as much interpreter overhead as possible, so if your choice of programming language is slowing down the JIT, that suggests that you have somewhere else to improve in the JIT ;)

Kernel/OS - This is the area that will have the largest performance difference between implementation languages, but with the major caveat that Kernel and OS emulation requires the least amount of processing power out of the 3 categories mentioned here. The Kernel and OS are responsible for managing and scheduling threads, handling network connections, and etc, but really most of these things have fairly low impact on final application performance in comparison to CPU and GPU emulation. As a side note, emulators aren't the only groups interested in switch OS/Kernel work. The open source Atmosphere custom firmware for the switch is working through recreating an open source kernel/os for the switch, and the two emulators benefit from their work too. (See the licensing exemptions here https://github.com/Atmosphere-NX/Atmosphere#licensing)

GPU Emulation - This is probably the trickiest part of Switch emulation, and once again, it comes down to how you emulate it, and not the language you use to write the emulator. The biggest performance differences between the GPU emulation of the two emulators will boil down to technique differences, and not the programming language. GPU emulation performance can be roughly broken into two parts, the "actual" gpu running time, and the state management/conversion. There's only so much an emulator can do about the actual GPU running time since at some level, you are going to need to run the game's GPU code, but the other half is a whole lotta code to avoid doing more work, and much of the GPU performance optimizations goes here. Things like managing the game's GPU memory, avoiding changing or querying GPU state unless necessary, converting nvidia shader ASM into SPIR-V or GLSL, and so on, are not generally going to be bottlenecked on the emulator's language of choice, but on the algorithms and designs that you use. Also a side note, the average comment about how "easy" switch emulation is because of "off the shelf nvidia parts" really misunderstands just how much work goes into this part. Switch emulation benefits greatly from the open source nvidia reverse engineering efforts from teams like nouveau, and others working on open source GPU acceleration on the switch like https://github.com/devkitPro/deko3d but also a great deal of effort from the switch emu devs themselves, writing tests cases to run on the switch to find edge cases and document behavior. It definitely is not easy work.

At the end of the day, every drop of performance counts, but some drops are much much larger than others. As such, the advantages of any language's performance characteristics will be heavily offset by the design choices the emulator uses. The creator of ryujinx is very comfortable with C#, and with good development practices, there's no intrinsic reason that one cannot achieve good performance in a C# emulator. And if one decides that it's worth the tradeoff to do some extra work for performance in exchange for a more comfortable development environment, then I say let them do what they want.

Shoutouts to both the yuzu and Ryujinx teams for all their hard work. I loved working on emulators a lot, and highly recommend anyone who's interested in contributing to give it a shot, its a really challenging and rewarding kind of project where there's always something new to learn about in a broad array of subjects.

redream is a Dreamcast emulator written in C only. libretro (the core API behind Retroarch) and Retroarch itself are primarily C. mupen64plus is in C, although the common 3rd party plugins and frontends are not always in C.

I don't think that was the case for me at least. I felt like (using your own analogy) the more I studied Rust, the better I learned how to use every other hammer. Learning a new programming language helped me learn c++ better since I was constantly on the lookout for potential memory leaks and other common pitfalls that Rust prevents. Now when I code in c++, I always try to write the code with safety in mind.

As mentioned elsewhere, its a combination of factors at least for here in Provo. Provo does have a sizable number of tech startups like LucidChart and according to this report it was number 11 on the tech startup list http://www.heraldextra.com/news/local/moving-up-provo-orem-a...

Also, as part of the deal for coming to Provo, Provo city sold the existing fiber network to google for a single dollar and google's job was to upgrade the system to gigabit. The whole infrastructure was in place for the most part, so it was a good deal that they couldn't pass up. http://www.tomshardware.com/news/Google-Fiber-Provo-iProvo-P...

Probably not, since this isn't a case where my co worker can just copy paste my code and hope it works for two reasons. The first is I am not entirely sure their testing environment is using 64 bit linux, it could be 32 bit linux since I'm not in the class, I just assumed its 64bit so I could test it on my machine. The next reason is they don't actually have a Hello World assignment at all! The reason I wrote this article is so I could share something I learned, and my co worker (who has already completed the first half of the semester's assignments) could use this as a starting point for how to write his own version for an assignment.

I went to download it from the eShop about an hour ago and it said that its currently only available from your local game store. I'm pretty sure that means Nintendo pulled it because of this vulnerability, but hey, I would probably pull it as well if I was Nintendo. I'm curious to see if its available for purchase for other people besides me.

HippyVM 12 years ago

For those that can't find the repository, it can be found here on github https://github.com/hippyvm/hippyvm I can never seem to find the link to github from the hippyvm page. It doesn't seem to be very active as far as emerging technology goes, but in my opinion it has good potential since it can build off existing language development tools that are continuously improving.

Discovery of Rust projects is super hard right now.

Good point! But I'm not sure of any other way to get projects noticed besides word of mouth and posting about them on sites like Hacker news and such. In fact that is where I first heard about rustful.

didn't know about your framework

My framework is actually at https://github.com/jroweboy/oxidize/tree/incoming (master branch is sorely out of date) and I feel I still have a long way to go before I get that finished :)

For some reason, no one has mentioned that there is a very usable rust REST framework right now that you can try out https://github.com/Ogeon/rustful I'm not the author of this, so I can't tell you what features are missing, but the features that are there work very well from what I can tell. When it comes to web frameworks in Rust, there aren't many options yet (as stated in are we web yet) but the few that are out there are in active development.

I agree that it would be more difficult for those kinds of games, but I think with a little bit of cleverness it can still be managed. Since it doesn't really matter where you split, but rather that you consistently split at the same spot, I think it can work by splitting on "You've got item!" text or other things like that. For instance, the first split of the Ocarina of Time escape could be split as soon as it displays part of the map for the zora river. The bottle split could be hit when you hold the bottle over your head, it could match part of the blue text box displayed over your hand since this exact spot on the screen would look the same for all N64 versions of the game. One of the key features of the auto split program would have to be that the image matching program wouldn't be so strict as to require pixel perfect matches, but also would be strict enough that it wouldn't pick up any false positives.

For segmented speedruns, you are correct, but more commonly what someone means when they say split is the speedrunner has a separate program running on his machine with a list of key points of progress in the game and the runner previous best times to that spot. When the speedrunner wants to race against his/herself, the speedrunner can start a split program with their previous record and play the game through. When a key point is reached, they press some button to denote that they reached this check point and the program will mark the time it took to reach it and tell you whether you are ahead or behind the previous record. Speedrunners use this to gauge whether they want to continue trying to beat the record or to restart and try again.

While I agree that the original source code for the game is probably long gone, I think we can settle for having a disassembled version of it. As you pointed out, the game was most likely in handwritten assembly, so the assembly output probably isn't much different. The most useful part would have been the comments, which it turns out there were some very dedicated people that went through the entire disassembly and annotated the source code like this one by doppleganger http://www.romhacking.net/documents/344/ and this one where movax tried to make the assembly more natural http://www.romhacking.net/documents/635/

There are several really cool things you can find on here about older games including a detailed explaination of the minus one world http://www.romhacking.net/documents/343/ where doppelganger covers exactly why the game does what it does.

I've had a similar project floating around in the back of my mind, except instead of reading the state off the emulator to determine when to split, the program watches the video stream and uses visual cues to determine when to split. As a simplistic example, consider the original mario game. When a runner would want to split, the game screen would display the text 1-1 with a black background roughly at the center area of the screen. With this information, I'm sure it wouldn't be too hard to determine when to split by scanning in the rough area for a picture similar to the what you would expect at this spot. I know that in practice this is possible to do since I have (just for fun) written a similar program for pokemon games when I wanted to EV train on an emulator. It would scan for the edge block of the tall grass (since all grass looks the same), and then turn around and scan for the edge again. When the screen went black, I knew I was entering a battle, so I was start scanning where the wild pokemon's name was and the program would be able to decide if I wanted to kill it or if I wanted to run (so I can keep count of the EVs I've gotten) The main issue with that program is that it couldn't tell shinies apart, so maybe over night I could have lost a shiny without ever knowing.

Anyway, long post short, I think it would be possible to monitor the live video and determine when to split based off information that can be stored in some kinda downloadable database (or some way to preset all the different required split information).

Oh thanks for clearing that up! Is there any specific reason why the rust compilier currently doesn't check that a static int accessing into a known length slice doesn't result in a compile error? (I'm only referring to cases where the index you are trying to access is known at compile time as well as the length of the slice)

Thats the rework that I'm working on! I'm debating between making a context trait and having the static function signature include a context instead of a request and the everything request related would be included/appended in the context, or having the user implement "controller" trait on their struct that has a method I will call after settling the routes (something like https://github.com/jroweboy/oxidize/blob/incoming/tmp/matchb... where maybe a macro can generate this method from the urls. Note that this code doesn't compile though :) While I was initially fond of the latter, it's been very painful to work with in practice and I don't see it actually working, so I think that I'll be testing out to see if I can rework the static functions to have a context. I think the hardest part is that I want to eliminate any need to have any static variables, but that means I have to store them all in a struct that requires Send to be fulfilled and thats more over head for programmers. In the near future, I will be looking into using the different parts of rust-http directly rather than implementing the server trait to see if I can gain any benefits from that :)

index out of bounds errors are handled in separate ways depending on the kind of array you are using. If I understand how it works correctly, you can be working with one of three main kinds of arrays: slices with a known length, slices with an unknown length, or a growable vector. In the case of slices with a known length, ie: let a = [0]; then trying to say a[1] is a compile error since the compiler knows the length of the array at compile time, it will not let this code compile. In the case of an unknown length at compile time, the compiler cannot really help you here, but it will cause a task failure when you try to access out of bounds (since it does know the length at runtime, it checks to make sure that it is in bounds). This is similar to what happens when you are trying to access an element that doesn't exist on growable vector as well. I hope this information clears up somethings for you and even more so I hope my information is correct! I'm still learning rust, so I still have misunderstandings quite often.

Kinda, yes, but not really. Currently, the best http framework in Rust is rust-http (https://github.com/chris-morgan/rust-http) and it has the things needed for handling requests and sending responses. But as far as actual full featured web frameworks, there is only one that I know of (discounting mre since it hasn't been updated in a long time) and that's because I am the creator of it. oxidize (https://github.com/jroweboy/oxidize) aims to be an inherently safe and fast web framework inspired by several other frameworks, but I don't consider oxidize to be even in a pre alpha state right now (pre alpha to me means, don't try to write code with it cause it will change drastically in the near future). I've rewritten the code for it twice already and a third time is sitting in a branch that doesn't compile right now. Progress has been a little slow lately since it became a group project for a class of mine, and that meant that I needed to manage several people working on it rather than actually contributing code. Over the summer I hope to get it in a good state and then make a post about it and fill out documentation, make tests, submit it to techempower benchmarks, and other necessities.

Having said that, I'm not confident yet that you want to be coding a web site in rust. In the name of safety, rust's compiler can be very difficult to please and in the name of speed it doesn't compile blazingly fast as well. Some people may have a different opinion from me, but besides writing a web app to learn the rust language, what advantages for the web does rust give you that go or haskell doesn't? When you are writing a web app, developer productivity is priority number one (in my opinion), and having to fight the borrow/lifetime checker and trying to find out why your struct doesn't fulfil `Send` are not things I would like to have to do when I'm making a web site. The tradeoff will be that hopefully a web site written in rust will be both fast and concurrently safe, but you will be sacrificing developer productivity for the reasons mentioned above. Irregardless of those issues, I'm still very determined to make a rust web framework that is both fast and easy to develop for (probably by providing examples of common web idioms in oxidize so people can see how to do it), just, I want people that use my framework to be aware of the tradeoffs that they will make. In short, I consider oxidize an experiment in trying to make a powerful, expressive, and extensible web framework in rust, and along the way I've started to question whether this will have any practical applications, but in spite of that, I still am trying to achieve the previous listed goals.

Rust 0.10 released 12 years ago

As far as fixing lifetime errors is concerned, I would recommend breaking your code down into a simpler form so that you can look at the lifetimes from a broader perspective. If you have a simple test case that mimics the same error that your main program has and you still can't solve it, you can simply make a gist for it and ask on the IRC and they usually are able to point out the error. In my experience with lifetime errors, doing it in this method sometimes reveals the simple error I was over looking, or in other cases after asking on IRC I learn something new about lifetimes.

Rust 0.10 released 12 years ago

One thing that has really helped me is picking a project that I really wanted to make and learning Rust while making the project. A read through of the tutorial also went over my head at first, but when the concepts started to show up in my program the tutorial also started to make sense. There is the documentation which you can also look through for specific examples, and also there are lots of small resources to read here http://static.rust-lang.org/doc/master/index.html Probably the best resource though is the IRC where you can ask just about any question and they'll usually answer it in a very short time.

Well, in my defense, I know what it means to be groovy and sorry if post made it seem otherwise. What I was misunderstanding was the fact that you can actually endorse personality traits on people since I have made a point of staying as far away from LinkedIn as possible. I thought that you were only able to endorse skills, and so when I saw they were endorsed for a personality trait I was a little confused and wondered if they meant that as a skill trait :)