This is probably going to end up being the longest post I’ve written so far. It’s also the most emotionally draining one I’ve written. I understand that not everyone will agree, but I ask that if you do comment, keep it professional. It’s already hard enough to write this and follow it up with the possibility I’ll be personally attacked or be even more emotionally drained because I had an opinion. Honestly, I would expect the Rust community would not make me worry like this, but that has unfortunately not always been the case. Also for the love of god do not post this on Hacker News, I’m trying to enjoy the last day of 2018, not have a bunch of people breath down my neck.
This post is gonna be as much personal as it is both technical and people oriented. Buckle up, we’re gonna cover a lot of things, and a lot of it is not fun. This is definitely one of those, “We really need to look at ourselves in the mirror” hard truths posts. As I type this though, I’ve reflecting on the past few years with the Rust community I’m just in awe of the amount of work, dedication, and passion that’s gone into it. I started Rust when I was still starting my career and finishing up my CS degree and now we have a brand spanking new edition! So much has changed that it’s honestly mind boggling thinking about it. Even at work I gave a history of Rust presentation and it was wild being able to go “back in the day” for something that was just a regular day a few years ago.
Now I find myself thinking about the problems we face as we become an even bigger community. We have a lot of problems. They’re good problems to have, but problems that must be solved eventually. I’m gonna go through all of them, what I see as major pain points/issues we’ve solved up to now, the present problems, and ones on the horizon. I’ll cover what I think we should focus on in the upcoming year and where we can go towards for Rust’s 2nd Edition. Let’s get started!
The Problems Behind Us
As I look to our future I’ve been thinking a lot about where we might go and where we are now and I feel uncertain and anxious. It feels like the same uncertainty and anxiety I felt back in 2015 when Rust 1.0 came out. What I was uncertain about then is not what I feel uncertain about now. In fact many of those doubts have been answered:
- Will anyone even use this? Yeah AWS, Google, Mozilla, and more companies, some of who we could not even imagine
- Would all of our effort be worth it? It’s tough to work on things and then not have them pan out. It sucks! Now though? Lots of people love and use Rust and those numbers have only been growing.
- Would this become easy for new people to use? Rust isn’t without it’s tough points and learning it isn’t easy, but I remember the Eldritch errors of old.
rustc: “You have a borrow check error!”
- Me: “Thanks
rustcwhere? What’s the problem?
rustc: “You have a borrow check error!”
The oldest amongst us might even remember the Lovecraft quotes that used to be a part of the many common ICE messages pre 1.0. Needless to say the developer experience is better overall for on boarding new Rustaceans. I’m not worried about this as much as I used to be, but like always we can spend time improving this.
The Problems in Front of Us
The big problem? Where do we go? So much of our energy has been spent figuring out how to get here that we haven’t thought where to go next. It’s like we’ve graduated high school. We’ve been through some tough times, but now what? Do we tread the path of yore and go to college or do we strike out on our own? There’s a lot that we could do! I often think of an excerpt from Sylvia Plath’s “The Bell Jar”:
I saw my life branching out before me like the green fig tree in the story.
From the tip of every branch, like a fat purple fig,
a wonderful future beckoned and winked.
One fig was a husband and a happy home and children,
and another fig was a famous poet,
and another fig was a brilliant professor,
and another fig was Ee Gee, the amazing editor,
and another fig was Europe and Africa and South America,
and another fig was Constantin and Socrates and Attila and a pack of other lovers with queer names and offbeat professions,
and another fig was an Olympic lady crew champion,
and beyond and above these figs were many more figs I couldn't quite make out.
I saw myself sitting in the crotch of this fig tree, starving to death,
just because I couldn't make up my mind which of the figs I would choose.
I wanted each and every one of them,
but choosing one meant losing all the rest, and, as I sat there, unable to decide,
the figs began to wrinkle and go black, and, one by one,
they plopped to the ground at my feet.”
I think it’s exciting to think of where we could go, but we’ll need to choose going forward, and that unfortunately means not prioritizing things some of us want, lest we end up as a wrinkled black fig of a language. We only have so many hours in a year. Less so given many are volunteer hours. This is the biggest issue we face, thus the call for blog posts that this post is a part of, and I think it’s going to define how we think of the problems ahead of us like a snowball turning into an avalanche.
The Problems Ahead of Us
Rust is in production now at some major companies. I’m fortunate to be working at a place that also does Rust, but this means there’s going to be some new challenges as we move into the next stage of our evolution as a language and as a community. What we’re gonna face will be uncomfortable. Rust isn’t just for hobbyists anymore and part of that means growing up as a community and answering some really tough questions. Let’s dive into it:
We’re gonna have some major players try to start giving us money as a community. Which is great that these companies see that they need to give back for the work of the projects that they use. Unfortunately, if you’ve ever looked at any other language that’s had to handle this it gets unwieldy. Politics start coming into play, what was once a fun hobby side project becomes enterprise software, or how we allocate scarce resources just becomes a not fun thing. I see this as a failure of Open Source to develop a good model to pay people for their work. We not only lack community, but legal infrastructure to do this properly, in a way that’s fair and equitable, but does not also make it hard to work together in the open on awesome software that helps us not reinvent the wheel all the time.
I had a conversation with a coworker and they mentioned “Why can’t we just tax software companies and pay people that way?” Then it dawned on me that Open Source work is what’s known in economics as a public good, something everyone benefits from, but one which many people don’t want to pay for it (I’m reeeeeeeeally generalizing here, but I hope you get the gist). Examples include things like roads, parks, infrastructure, garbage services, fire fighters, etc. We benefit from them, but privatized versions of these are terrible. Could you imagine paying for every road you took? There’s no way to maintain their utility to society outside of the government maintaining them, thus public goods. However, getting this model of public goods to work with open source is, well difficult if not impossible. Look at foundations, supposedly stewards of various language’s work and projects and all of those end up failing spectacularly in some way. We need something else.
New People/Stake Holders
Great news we’re successful finally and we’re gonna have a whole lot of new people! Bad news we’re now gonna have a whole lot of new people. This seems paradoxical that this is good and bad, but it’s not! We’ll have lots of new people to work on features we’ve wanted, we’ll have more awesome talent to learn interesting things from, we’ll have people with insight from different communities and walks of life and all of this is awesome! However, with new people comes people with different priorities and wants, and those might be orthogonal to the trajectory we’ve been on or what we as a community want. This does not mean we should not listen to them! Quite the opposite. What it makes more difficult is gathering consensus as a whole. It’s like CAP theorem but for RFCs and people.
So like I said new people is good, the things it makes more difficult has already been a hard people problem. Just look at the infamous
non-ASCII Idents RFC and now compound that with way more developers and companies with their own agendas. Again not a bad problem to have. It’s just a difficult one. The solution will likely be giving teams more autonomy to do what needs to be done without too much centralized management, but we’re laying out problems not solutions right now.
Maintaining and Upholding Current Community Values
Which leads us to the next hard part about having a whole bunch of brand new people. How do we maintain the values that have been a core part of Rust and why people stick around? Eventually we’ll have people who use Rust just for their job, but they might not like or gel with things like the Code of Conduct.
Why wasn’t I consulted?
The most recent example of this? The website. And it’s only going to get worse the more people we have. Real talk, the bigger we get the less we can satisfy everyone. Remember the poem I mentioned above? Yeah well this is one of those cases. We needed to revamp the website, but we can’t have everyone do input and ship it on time. Not saying we can’t improve things over time, but just realize we need to make a decision at some point and execute, and that means some people are gonna be unhappy. It really only gets worse as we get bigger. However, I think the core team and us as a community have the right ethos overall. See this thread on the scaling issues around an open community and this talk by Niko and Aaron to understand what I mean.
This is a hella difficult problem to solve. Striking the balance between community feedback or analysis paralysis. We can debate for years, but time continues to move forward. We can have solutions that are available or try to gain consensus, but having both is hard. I think going forward we need to have empathy. A lot of empathy. Remember we have others in our community sitting behind those screens, trying to do the best they can for all of us. We all want Rust to succeed and if we continue to look at problems only from our point of view, or if we throw around criticism without care, “Wow this sucks” we will end up poisoning this community from the inside out. Good people will leave. Frankly that’s the thing I'm most worried about. Toxicity brought about from being truly open and vulnerable to criticism.
If your first reaction to that is grow a thicker skin you’re not the kind of person I want to work with. I already have enough people like that I have to deal with in the military and frankly they suck and only make me want to leave sooner. No one should have to subject themselves to abuse just to participate in a project. I worry as we grow this is gonna happen more and more as unsavory people from other communities where this is the norm join.
“Michael that won’t happen!” Trust me there’s a reason I don’t post/comment on /r/rust as much and have instead gone to lurking as the default. It changed over the 3 years I’ve been going to it. So too will our community. The question is what we’re gonna do about. We have a chance here to really carve out a fantastic place for people in OSS in a very visible way. I’d hate for us to really mess that up.
Quality of Libraries
We’ve got some really great libraries (Serde, Diesel, Hyper, etc.), but we don’t have a lot of 1.0 libs, nor is every library that one uses guaranteed to be supported long term, which, hey guilty as charged. I haven’t maintained a lot of my libraries for a multitude of reasons. What can we do as a community to be a steward of libraries long abandoned by their maintainer? Just look at web frameworks like Iron and Nickel that some of the earliest Rust users are now stuck with. What can we do to help maintain quality in the ecosystem so that anyone can participate, but that the overall bar for quality is high? I think Liblitz in 2017 was a good start, but it’s resource intensive and overall we only got some major crates shipped at 1.0. I think we can learn from this and make some real strides here. The main thing that will keep projects going is getting them to grow the people to maintain them and giving ownership to those who can and want to do it. There’s a lot we can experiment with here and I’d love to see what we can come up with to make our ecosystem vibrant and healthy and one that does not burn people out and spit their dried out husk into an ever increasing pile of abandoned crates.
I’m lucky in that I get to use Rust at work. Unfortunately with a 125K plus line codebase (more with ridiculous amounts of proc macros and for good reasons) I face every limitation of Rust daily. It’s clearly built for those of us who use small code bases. Debugger support is okay at best, laughable in general, and awful in most cases. I don’t mind using
println, but the fact it takes 14 to 15 minutes to compile all the code just to print what’s going on is horrendous, with god knows how many hacks to increase caching. This was not an issue as a developer on my own code base, but in production? Wow it’s noticeable. I get all the complaints from 2015, 2016, 2017, and now 2018. Seriously broken code between minor pre 1.0 packages is common (but totally expected), building on Docker is really hard to cache already built deps (this issue has been open for awhile) and we had to do some weird hacks to get it it to cache. I’ve seen some things that really have pushed Rust to it’s breaking point and it makes the need for things like GATs and other things we already have agreed to make via the RFC process more pressing. Simply put we gotta stop coming up with hot new things and polish things that are already out, or begin the foundational work on features we agreed too.
Cargo feels great if you only have a rust project, but if you don’t you either thread everything through
build.rs or you end up invoking
rustc directly and neither is ideal for production needs. Having interoperable ways to work with other build systems well would be great.
Simply put: clean up what messes we have, make what we planned to, and polish the experience.
Rust 2nd Edition: Enterprise Grade OSS
I first want to talk about what I mean for our next edition to focus on “Enterprise Grade OSS”, before moving on to Rust 2019 Roadmap goals, because these long term ones have molded my thoughts on 2019 and what we should do this upcoming year.
First up: Open Source Software (OSS) is absolutely garbage and unsustainable as the world currently knows it.
“Michael that seems like a pretty nuclear take.”
It’s not actually. In fact, in the problems ahead I already covered the main things that are going to be a problem, and are the fundamental root of OSS being flawed, particularly volunteer work and extraction of labor from volunteers being the basis, and OSS self selecting for particular kinds of individuals who have “thick skins” or have the ability to work on things for free for long periods of time. Not that there’s not compensation in exposure, but if those hours had an actual price tag for dev time associated with them, we’re talking millions of dollars for the past 3 years alone, and that’s just the engineering cost.
Worse this means Rust is less accessible (in the sense anyone can contribute and join without financial difficulty) and diverse (people from all walks of life and backgrounds able to contribute) then we would want it to be. I’m happy to have participated in Rust Reach, but there’s more we can do to get people involved and help them along.
Secondly: Our organizational structure has hit it’s breaking point. I feel withoutboats covered this well and while I have touched on it here, their post is pretty comprehensive and hits all the key points I feel about it
Thirdly: Our behavior as colleagues needs to grow up. I’m sick and tired of seeing people who give a shit, who do the hard emotional labor, who work tirelessly for this project, get hurt and drained constantly, because well I don’t like this etc. etc. The behavior does not breach the Code of Conduct, but it very well violates the spirit of it. I would not tolerate this kind of behavior with my own coworkers, yet we constantly do here. We’re killing each other with paper cuts by producing the same endless criticism. I’m tired of it and I’m tired of caring. 2018 has already been bad enough politically here in the US and to see similar things, while not as overt, creep into Rust spaces is killing my soul and others who care.
What does that mean for the next edition? In short:
- Find ways to deal with money and transfer of value that’s not a foundation, whether through bounties for issues from anonymous companies or people
- Find ways to scale management of the project in general and fix the underlying structural issues before it comes tumbling down
- Create a Professional Code of Conduct for how we treat each other in an official capacity that is more strict and supersedes the Code of Conduct used in informal settings
- Create a scalable decision making process that isn’t the current RFC process
- Provide ways to fund and/or provide equipment for people doing critical work that isn’t the current model of OSS
- Increase our ability to get more contributors involved with a lower barrier of entry
- Polish existing code, pay off tech debt, and begin work on already agreed to things
- Limit and be more conservative of accepting new ideas and instead defer them to a time we can work on them. Maybe a deferred accepted RFC.
- Make Rust a beacon of what OSS could be industry wide
- Choose community/people over corporations for major decisions
- Fix production needs (but corporations you say! Yeah, I know. This is a very hard fine line to balance)
This is ambitious. I truly believe if we don’t the project will fail in an irreparable way, making it hard if not impossible to ever fix as the costs of doing so grow larger. For too long we’ve sacrificed too many people on the altar of tech for the advancement of a few in the pursuit of material goals. I want us to instead nurture ourselves in the pursuit of goals that benefit people. The tech will follow.
Rust in 2019
With those over arching goals in mind what should we do in 2019? Here’s my list in service of those goals
- Freeze new tech ideas for the year. We should polish what we have and solve more pressing issues
- Fix our organizational and structural issues such as the RFC process and team organization
- Adopt a professional code of conduct for official Rust spaces used to do any sort of work on the project, or in service of it
- Figure out how we handle money and funding with limited resources going forward in a way that’s not basically unpaid labor that might later pay dividends by getting a job in Rust
I know these are vague but I see these as the most pressing in the upcoming year to lay ourselves a solid foundation for the next three years. I have some ideas for these things, but writing them out I think is better served in a discussion process if we even settle on these ideas for the upcoming year.
I think this post was pretty bleak and depressing sounding. Frankly it’s meant to be, not because it means we should not and can not fix the issues facing us, but because we should and can fix them. We’ve made our cake and ate it too on a number of occasions, and I think we can here with sustainable open source software. Come with me this year and help build the project and possibly the world we want to see. I promise you if we do, you won’t regret having been a part of it.