Vapour: A typed superset of the R programming language

by johncoene • 2 years ago

83 59 2 years ago

brudgers2• 2 years ago

[flagged]

johncoene• 2 years ago

First, how is that "giving myself an excuse"? Second, it's a total non sequitur, and even then, it's a day old has it broken?

brudgers2• 2 years ago

the syntax might change, things will break, expect bugs.

Bugs are normal software development.

Changing syntax and breaking things make work for everyone else for the convenience of developers. Reliability is what makes a tool a tool.

maleldil2• 2 years ago

I have no skin in this game, but why are you expecting maturity from a new language? They're not claiming to be production-ready – quite the opposite. It's on version 0.0.5, for goodness' sake.

lloydatkinson• 2 years ago

This looks nice. I find R to be an unreadable mess. The comprison shows a great improvement.

qudat• 2 years ago

The default IDE workflow is like a python "notebook" where code can and is run in whatever order the creator wants. Every R code I've read treats it as such and it results in an absolute mess to read and manage.

andrewla• 2 years ago

As an R programmer the examples given on the landing page seem very foreign to me -- you are almost always writing vectorized code in R, so I would think that would be front and center.

    let x: int = 1

Is this a list of ints or a pure singleton? R doesn't have scalar types, so it would seem the former, but the example makes it unclear. Later in the docs it makes it clearer:

    let x: int = (1, 2, 3)

And this, as an R developer, I can definitely get behind -- the c(...) syntax is always awkward and having a native syntax for static arrays is a welcome change.

juujian• 2 years ago

Yeah, it's not an idiomatic example. I like the idea, but this makes me worry that the project does not have the right priorities. I.e., supporting my use cases :D

Alifatisk• 2 years ago

How does a R developer differ from a R programmer? (Asking you as an R programmer)

ecshafer• 2 years ago

I think this is a great idea for the project. I don't dislike the syntax, but the syntax seems more ML than R to me. I think keeping the syntax more R-like could be worthwhile.

clircle2• 2 years ago

Statisticians and researchers, is this helpful?

tech_ken2• 2 years ago

I would say that vast majority of type problems in data science/stats workflows come from data tables "trojan-horsing" type or missing data issues, rather than type problems strictly at the code level. Type annotations won't help you when your upstreams decide they want to change the format of their year-quarter strings without telling you.

dragonwriter2• 2 years ago

Type annotations won't help you when your upstreams decide they want to change the format of their year-quarter strings without telling you.

IME with both Python and JS/TS, it helps a lot (which is different than completely solving the problem), for reasons which should generalize to other typing add-ons/supersets for untyped languages. Typing your code forces validations at the boundaries, which obviously doesn't stop upstream sources from messing with formats but it does mean that you are much more likely to catch it at the boundary rather than having weird breakages deep in your code that you have to trace back to bad upstream data.

levocardia• 2 years ago

Not really, because honestly a lot of us who came into programming via research never learned typed languages or unit tests or any of those best practices - we were just hacking around in MATLAB, R, or Python from the start. What I really need is a seamless and easy way to run statistical models that can only be fit in R, but from Python or Node. There are several categories of statistical modeling where R completely blows python out of the water, and it's incredibly wasteful (and error-prone) to try to re-implement these yourself in Python.

bachmeier2• 2 years ago

rpy2 can be used to call R from Python: https://rviews.rstudio.com/2022/05/25/calling-r-from-python-...

reticulate works for going in the other direction: https://rstudio.github.io/reticulate/

With the good interoperability these days, let's stop rewriting functionality in other languages. If the interoperability is no good, work on fixing that, please.

ellisv• 2 years ago

It is probably helpful in some cases and unhelpful in others. R uses multiple dispatch, so calling `foo` on different types can produce different output. It isn't clear to me how Vapour handles this. In general though, folks are passing around data.frame or similar objects.

rscho3• 2 years ago

No, it will get in the way more than anything else. As has been said elsewhere in thread, what we need to ensure in R is mostly runtime constraints (array shape, number in specific interval, etc.) this would require a super heavy and complex type system, with at least refinement and probably fully-dependent types. It would be too complex to use for most people and use cases. A contract system would be far more practical and useful. See CHECK and other constraints in SQLite. That's exactly what we need.

mushufasa• 2 years ago

The main reason we shy away from R for production apps is all the silent errors where things seem to succeed while being horribly wrong if you take a look. Typing would certainly help mitigate that.

uptownfunk2• 2 years ago

Will this fix the problems it claims to? The power of R is the rich package ecosystem. It caters to people who don’t want to think about engineering concerns but want a fast way to access the powers of computation rather than building a scalable system, two very different things. It excels at the former. A new language will not fix this, because this type of thinking has infected the entire package ecosystem. Frankly with code translation you probably don’t need a new language. Prototype in R and code translate to Python or whatever you want to use in prod. Or frankly just do code gen directly in Python so you can skip having to confirm if the results match.

To be clear, I love R, it excels in prototyping but I have seen too many real world struggles of folks trying to move to prod that I would say save it for EDA projects and one time analyses.

_Wintermute• 2 years ago

I often find I want a specific statistical package that's only in R, but want a more general purpose language for all the other stuff that's involved (parsing, filesystem stuff, error handling etc). I don't want to risk re-writing the statistical methods and all their dependencies in the sensible language, so I end up calling R only for the statistical methods, but I can see this as an alternative.

joshdavham4• 2 years ago

A new language will not fix this, because this type of thinking has infected the entire package ecosystem.

Do you think the culture of the package ecosystem could possibly change in the future?

uptownfunk2• 2 years ago

Even if it does the problem will still be there

bachmeier2• 2 years ago

I took a couple stabs at this long ago (even before there was a Typescript for inspiration). The first attempt was to add types to the syntax of R, but that would have required a lot more time than I had. Properly catching errors is a massive undertaking requiring a lot of background I don't have. The second attempt was to add syntax for types to R and then compile the code to another language. That's easy to do, but really boring, so I wasn't able to stick with it. It comes with the advantages of static typing and R code that runs very fast. I gave up and went with embedding R inside a statically typed language. Very happy with my choice.

Good luck to the authors of this. I believe it solves an important problem for R package authors and others wanting to write bigger programs. It's hard to argue with the benefits of static typing for this type of work.

layer82• 2 years ago

Sounds like vapourware. ;)

joshdavham4• 2 years ago

I mean, there is an alpha you can download. If it was just a landing page and an email waitlist, then that would be vaporware.

layer82• 2 years ago

I was commenting on the naming choice.

joshdavham4• 2 years ago

Looks interesting! What types of programs do you think people would write in this language? I don't see an obvious need for traditional R programs which are usually just scripts for working with data, but maybe people could write R packages in this language?

johnnybzane3• 2 years ago

How do I find jobs that use the R language? It's impossible to search the letter "R" on linkedIn or Indeed without getting a bunch of unrelated job postings

"R" is the only programming language I know and I can't find a job that uses a R because job search engines don't allow you to sort by skill

"R language" is the closest substitute on linkedin but the results are still a jumbled mess of jobs, some looking moreso for other skills (SQL/Python)

I know R-heavy jobs exist but finding them on LinkedIn is virtually impossible

clircle2• 2 years ago

Why would you do that? R is a just a tool for doing statistics or research. You need to search for jobs in your subject area like "ecologist", "econometrician", "green energy reseacher", etc.

johnnybzane3• 2 years ago

There are hedge funds that like hiring people who know how to manipulate data in R using dplyr and data.table

Looking for a similar job where my desire/interest to spend all day in Rstudio is a value add to a business

nerdponx4• 2 years ago

Because if you work on a team you need to use a language that the whole team can work with. If I'm the one R guy at a Python shop, it's not going to work out well. It depends a lot on org structure of course. But I think it's telling that the jobs you highlight are mostly academic jobs where the practitioner would be expected to be a highly competent individual working largely alone, or in a very small group, carrying out research on behalf of some stakeholders, and not likely to have to put anything "into production" any time soon.

For example, I used R (data.table) when I was a solo data scientist working on a consulting project where I needed to work with a dataset on the order of a few billion rows. I had nobody around to constrain my choice of tools, so I went with whatever felt convenient, familiar, and ergonomic for getting the job done.

Today, I am on a team of 5 other people, none of which know a lick of R, and my code needs to run in production pipelines that need to at least in theory be debuggable, auditable, fixable, etc. by people other than me. Therefore I use Python, because we are a Python team and that's the language that we use, end of story. (Python also happens to be a good choice on our team for other reasons, but that's not the point here).

Maybe the best industry where you are likely to find people doing "production" work in R is some form of insurance. But even back in 2017-2020, things were shifting towards Python at the one P&C company I worked for.

Balladeer• 2 years ago

How does "R language" compare to searching for one of the popular R packages? Searching for "tidyverse", "dplyr", or "ggplot" seems to get a good chunk of hits. That being said, yeah, there does seem to be a trio of skills that often go together (R, python, SQL)

johnnybzane3• 2 years ago

If you search specific packages on LinkedIn the number of jobs is usually very small

E.g. tidyverse or dplyr is like 20-40 jobs. ggplot is 88. There's definitely way more than 100+ companies looking for R-heavy users.

kagevf• 2 years ago

I tried using "r" (with quotes) on indeed, and got some hits where R was listed as one of the necessary skills.

dkga• 2 years ago

Perhaps #rlang would work? Or #tidyverse if you are feeling tibblish :)

russellbeattie2• 2 years ago

This isn't specifically about Vapour, just about what's become the common way to specify types.

I know this is totally bike shedding, semantics, vi vs Emacs, BigEndian vs LittleEndian and it's too late now to affect anything, but to me using a colon after the variable is just wrong!

let x : int = 1

func add(x: int, y: int): int { return x + y }

I see that and it looks like int = 1 and the function's return type is totally lost.

This seems completely backwards to me. Maybe I'm just used to the way C did it, but the variable modifiers should come first.

let int x = 1

func int add(int x, int y) { return x + y }

Why we reversed it and added in the colon just doesn't make much sense to me.

lrasinen• 2 years ago

": type" has a long heritage, going back to (at least) Pascal in 1970.

russellbeattie2• 2 years ago

I didn't know that! Still doesn't make much sense to me.

nerdponx4• 2 years ago

I have some questions that are not answered by the homepage.

1) How does this work with function parameters that are intended to be captured unevaluated with substitute()? Do you type the input as "any" and document separately that the parameter is kept "unevaluated" as a symbol/name or call?

2) How does this work with existing untyped R code? Does it at least include types for the standard library (or some subset thereof?)

3) Is there any type inference, or does it require explicit type annotation everywhere?

4) How do you propose to handle NA (which can appear "within" any typed vector)? Does the compiler support refinement types? If not, how does checking for and preventing nullability work, when checking for NA values requires a runtime check?

5) How do data frames work? Are they typed like structs?

6) Which object systems does it support, if any? S3, S4, Reference Classes, or the 3rd-party R6?

As much as I like static types, I feel like R is maybe the language where I need or want them the _least_. How often do you really run into a situation where you pass a character vector to a function that requires a numeric vector and it crashes your program?

99% of the time what you really want is known-valid data frames for data processing, and statically-sized arrays for math stuff.

fn-mote• 2 years ago

As much as I like static types, I feel like R is maybe the language where I need or want them the _least_.

I really disagree with this.

I think one of the whole reason there is a whole Tidyverse ecosystem that the behavior of (some) R code is unintuitive in a way that adding typing would absolutely improve.

It seems like you're deeply familiar with the R ecosystem, but as a user what I want is a safe subset of R that I can use.

How often do you really run into a situation where you pass a character vector to a function that requires a numeric vector and it crashes your program?

In R the more likely situation is that you pass in the wrong typed thing and it silently continues with very unexpected values being passed, causing trouble or errors much later in the program. Which is very much a problem that typing helps with.

nerdponx4• 2 years ago

In R the more likely situation is that you pass in the wrong typed thing and it silently continues with very unexpected values being passed, causing trouble or errors much later in the program. Which is very much a problem that typing helps with.

Can you name one practical example of this happening as a result of passing in a vector of the wrong class()/mode()? Not a data frame with the wrong column types, but an actual standalone vector. Can you name an example in the Tidyverse ecosystem that specifically improves on the type-safety ("class/mode-safety") of the standard library? I can't, but maybe that's just because it's been too long since I did anything serious with the language.

I can definitely think of complicated interfaces where you can silently get strange results by passing in the wrong thing. sweep() and apply() are obvious examples, where you can accidentally swap the argument order and silently get a nonsensical result. But that's a matter of array shape, not of type. Try passing an argument of the wrong type (again, where "type" in this case means the class or mode of the vector) to sweep() or apply(), and watch what happens: you get an error message informing you that you passed a value of the wrong type. At worst, you get an obtuse error message informing you that you passed a value of the wrong type, but bubbled up from some internal code. But you get an error all the same.

R is actually very strongly-typed, and abstracts over some details that would otherwise cut into that type-strength. For example, R doesn't have the Numpy problem of exposing different physical storage sizes for integers and floats! It just has abstract numeric arrays, backed by whatever the hell storage type the R language implementers decided to back them with, with no opportunity for the user to accidentally mix things up and lose precision, overflow, or crash on contact with some pre-compiled Numba function.

I maintain that a much, much more pertinent problem is that array shape is not part of the type system, and moreover that a lot of R code is (by design) highly polymorphic with respect to array shape, precisely because there is no such thing as a "scalar" number or string but we still want to let people use numbers and strings in scalar-like fashion.

NULL I think falls into this category as well. NULL in R is a bit like nil in Lua or undefined in Javascript, in that it has a kind of dual function as a "value that is not any other value" and a "non-value that cannot be inserted into a collection, instead deleting whatever was previously there". But when is the last time someone got a NULL and a numeric vector mixed up, and wasn't able to figure out what happened? Is all the complexity of a static compiler really necessary to catch that relatively rare mistake?

Maybe the one exception here is the factor class. But there's no mention of factors here, and (as with array shape), validating factor levels is probably more important as validating that the thing is a factor in the first place, as opposed to character.

The NA checking proposed is another story. Now that would be useful, but so would checking things like min/max ranges, the presence of certain columns in a data frame, etc. For example Python has its data frame input validation framework Pandera that offers at least some of these guarantees at the type level.

As for classes, I noticed that they implement what looks like a nice concise syntax for creating S3 class objects with structure(). That's great, but you could have just written a helper library to do that.

Anyway, here's a project where someone designed a whole language and wrote a compiler for it, and I'm just one cantankerous former R user doubting whether that project is ever going to be useful. If this is just a hobby project to scratch someone's itch: ignore me. But if this is intended to be a serious thing for serious use in production, then I'd encourage the creators to reconsider how they portray their value proposition, and to maybe reconsider whether the goal of their project aligns with the needs and desires of actual R users in industry, of whom there are still many, but definitely not as many as there used to be.

rscho3• 2 years ago

Static types seem like a bad idea for most R use cases. Contracts, on the other hand would be absolutely stellar. A-la-SQLite style.

nerdponx4• 2 years ago

Is there not a decent contracts framework for R yet?

condwanaland• 2 years ago

Cool idea! Looking forward to exploring it this weekend

wodenokoto• 2 years ago

I think this is super interesting. I’m not convinced by the examples that this is the right next step, but I hope that either this is alpha enough to experiment more and change direction, settle into a niche within R (like maybe libraries are developed in vapor and compiled and distributed in R?) or that this inspires more ways of supersetting or subsetting R.

xiaodai• 2 years ago

Never thought someone would do this for R. Really nice work.

Vapour is an interesting choice. Hope it’s in name only :)

data_ders• 2 years ago

Is it fair to compare this to what Typescript provides for JavaScript?

Vapour: A typed superset of the R programming language

Related Stories