kohsuke

You can make reservations online: https://www.eki-net.com/en/jreast-train-reservation/Top/Inde...

So they run 5 different experiments to test the hypothesis, and they were nothing like what I imagined.

For example, in one study, they divide participants into two groups, have one group watch https://www.youtube.com/watch?v=fn3KWM1kuAw (that highlights the high socio-emotional capabilities of a robot), while the other watches https://www.youtube.com/watch?v=tF4DML7FIWk (that highlights the low socio-emotional capabilities of a robot)

They are then asked if they agree or disagree with a (presumably hypothetical?) company's proposal to reduce employees' welfare, such as replacing a meal with a shake. Two groups showed a different preference.

This makes me think about that old question of whether you thank LLM or not. That is treating LLMs more like humans, so if what this paper found holds, maybe that'd nudge our brain subtly toward dehumanizing other real humans!? That's so counter intuitive...

But that's beside the point of the paper. They are talking about how the humans perciving the "socio-emotional capabilities of autonomous agents" change their behavior toward other humans. Whether people get that perception because "LLMs hack our brain" or something else is largely irrelevant.

I don't think the point of SLO = flakiness out of control. The point of framing it as SLO is the realization that neither extremes are good. Flakiness cannot be allowed to get out of control that some efforts must be spent to contain it, but it's also unnecessarily perfectionist and thus the waste of precious engineering bandwidth to eliminate them completely. The whole point is to avoid "bureaucratic games", as you call it.

My theory is that the lack of easy mechanism to measure the flakiness is stalling the progress. If the overall flakiness can be measured, and the top offending tests identified, then I think it becomes no brainer to spend efforts curtailing them back when the flakiness gets too high, but otherwise exclude flaky tests from, say, PR merge gate.

This is indeed a religion, because in my experience people tend to feel strongly holding very different positions. You can already see it in this thread.

I think quantifying and prioritizing is key, like you wrote. Respected engineering organizations like Google and GitHub all came to the same place. Flakiness is often unevenly distributed, so find & tackle ones that are the worst. Don't try to eliminate the flakiness because that's not economically viable.

I'm trying to put my money where my mouth is... we'll see how it goes.

I forgot which book it was (maybe "the three body problem"?) but there was a science fiction story where a Chinese king makes his soldier act as a logical gate and his army becomes a computer. I was like, wow, I didn't think about that, but it totally makes sense!!

Thanks for taking time to put this together. Yes, much of it isn't new, but it's always good to hear how these dots are connected in other people's view to form a theme.

On C, I think we've made a good progress <https://github.com/jenkinsci/configuration-as-code-plugin> that I think you'd like.

Hi, I'm Kohsuke, the creator of Jenkins. I'm sorry to hear that you had a bad experience.

Would you be willing to letting me interview you so that I can learn where it failed your expectation? I'm honestly trying to learn where we can do things better, and often what's obvious to one person is completely incomprehensible to another. So I think this is a great opportunity for me to learn a fresh perspective.

My contact information is in my personal profile.

I recommend https://www.adachi-hanga.com/ukiyo-e-en/ What they sell are not poster prints, they are made the old fashioned way: https://www.adachi-hanga.com/ukiyo-e-en/quality/index_en.htm...

Thanks for your thought. I took your main question to be "why bother?"

I think a part of it is that I fundamentally believe in an extensible system. The world of software development is so diverse, and we have smart people everywhere. So I always felt that the best thing a geek like me can do to other geeks is to give them a shoulder to build on. I don't think that's a solved problem, and to me, that'll always be an important value of the Jenkins project, more so than any code.

I think a part of it is the responsibility to users. Jenkins is very widely used software, and it's an incredibly important part of the software development process for many. I appreciate that kind of trust, and I want to deliver better software for them. I think people in the community shares the same passion.

As CTO of CloudBees, serving our users and customers, and broadening the adoption base are an obviously important business goal. So the interests are aligned there as well.

And finally, I think this kind of "reinvention of the brand" happens all the time. Windows got reinvented from 95 to NT, Firefox got reinvented a few times. There are many other examples less famous but closer to my part of the universe, like Maven 2, GlassFish 3, ...

Exactly our thought! The whole Jenkins Pipeline is around the notion that your job definition should be version controlled (and you don't necessarily have to lose GUI, see our "blue ocean pipeline editor" that now comes out of the box.

Then the newest kid on the block is https://github.com/jenkinsci/configuration-as-code-plugin, which I referred to in my doc.

Thank you for the encouragement.

Indeed I wanted to capture the shortcomings correctly, because I truly believe in the power of the community to solve them. Looking at this thread, I feel my summary is largely validated.

We've been working to solve those, and we'll step up even more so. Exciting times!!

I'm sorry to hear the bad experience.

I recognize those challenges in my pitch, we have various efforts already under way to address them, and with this gear shifting, I think we'll be combining those in a compelling way.

For example, defining Jenkins config in YAML in Git is a key piece to solve a fear of config change, and this is called "Jenkins Configuration as Code" and is under way for a while now.

Cloud Native Jenkins will also split single process "master" into many build-as-a-function kind of processes, so it isolates builds and allows changes to be rolled out more incrementally.

There's more focus on us owning a bigger responsibilities around "basic every day stuff," too.

Author of the post. I agree that it is our biggest strength & weakness, I acknowledge that problem and put forward some solutions in the doc.

One piece of the solution is to embrace core and a bunch of important plugins together as the foundation. Normal users shouldn't be asked to pick & choose the basics like that, and we want to lock down the combination of the versions in that group. Whether those are behind the scene plugins or not from contributors' perspective is an implementation detail.

Another piece of the solution is to grow more extensibility mechanism beyond the current in-process plugin. There's a thing called "Pipeline shared libraries" in Jenkins, which is a good example of this. It lets developers create higher level pipeline primitives by composing other existing ones. There's some mechanism to share those with the community, too, although not as sophisticated as plugins. From users' perspective, it extends capabilities of Jenkins just like plugins, but in a way that doesn't create the kind of instability a bad plugin can -- its impact is local to one build, for example.

Then there's the container-as-a-building-block extensibility, Jenkins Evergreen, and more...

You should be presenting in a future Jenkins World event!

With "declarative pipeline" introduced a few years back, this is the direction we are pushing people toward.

Programming capabilities are useful for ecosystem developers to create higher level primitives from existing ones, as it creates a new way of extending Jenkins without plugins.

I'm Kohsuke, the creator of Jenkins and the author of this post. Happy to answer any questions people might have.

Thanks, this is really helpful.

I need to find someone from the pipeline team to pull into this, so in the mean time, just responding to what I can contribute,

Just to make sure, I'm not saying parameters are disappearing. I'm just making an observation that less people seem to be using it. Take your example of release vs general checkin/PR builds. I see increasing people doing releases as automation that kicks in after creating a tag, or by cutting a release branch. Or the master branch is always deployable.

I agree with you that reporting capability needs to be better in order for one Jenkinsfile to pack lots of different test cases. I believe the team totally gets this importance.

The "system config DSL" you mention has evolved into "Jenkins config as code", and I've referred to it in my other comment. I think we are on the same page that it's a crucial part of a repeatable mature Jenkins setup. I think I also totally get what you mean by "unwieldy", and Jenkins Essentials in that comment is making steps to attack that challenge.

I'd love to hear from you where Jenkins X fell short for you, because I think it should speak to some of the challenges by embracing a certain best practices.

First, full disclosure, I'm the creator of Jenkins.

I'm sorry to hear that you had a bad experience with Jenkins Pipeline. I can see that you know a lot about it.

There's actually no difference between how continuation is handled between scripted and declarative, so I'm curious to know more about what hit you, because I suspect it's something else (though obviously equally frustrating!) Similarly curious about improvement to error reporting, because I think that's something the Pipeline team cares about, and it's one of those things where how errors in real world are made is always more interesting than what we can imagine. I used to work on compilers, so I know the frustration of a poor error message pushing you down a wrong lane, only to discover a few hours later that all it took was one line fix! Modern improvements in Pipeline (like declarative and this one) is in no small part motivated by making those error checks more thorough, easier, and more upfront. So I think this is a change in the right direction.

My perception has been that parameterized jobs are in decline, in part because more people are triggering automation implicitly through commits, and not through explicit "run this" button. Parameters are more implied from the context (branch, commit message, creation of tags, etc), as opposed to explicitly given from UI.

Stepping back from those specifics, I think regrettably software has bugs, and there's always more usability improvements that can be made, so we are just working on those one at a time, which kinda summarize my entire journey with Jenkins :-) So in that spirit, I want to make sure we learn from your suffering.

Creator of Jenkins here.

First of all, this is a Google Summer of Code project. Abhishek, who is driving this work, is doing a great work, so I hope people can give him encouragements and feedbacks to push him forward. I'll make sure he sees those feedbacks and will stop by to answer questions you might have.

This is one of the efforts that is pushing the envelop of Jenkins that solve problems people had with Jenkins. Reading some of the reactions here, I wanted to use this opportunity to introduce other bigger efforts going on currently in Jenkins that I think addresses various points raised in this thread.

* Jenkins Essentials is aiming to be the kind of "readily usable out of the box" Jenkins distribution that is a lot less fragile, because it's self-updating appliance that has sane defaults and obvious path to success.

* There's architecture effort going on to change the deep guts of Jenkins so that data won't have to be on the file system, and instead go to managed data services.

* Jenkins configuration as code lets you define the entire Jenkins configuration in YAML and launch Jenkins as a docker container to do immutable infra. Jenkins Pipeline lets you define your pipeline in your Git repo, so that's the other part of immutable infra, and between modern pipeline and efforts liek this one, there's no need to write Groovy per se. It's just a configuration syntax based on brackets like nginx, which happens to conform to Groovy syntax, so that when you need to do a little bit of complicated stuff you can, but you don't need to

* Finally, Jenkins X is focused on making CD a whole lot easier for people using and developing apps for Kubernetes. It's a great example of how the community can take advantages of the flexibility & the OSS nature of Jenkins to the advantages of users.

* A few people mention about container-based build environment, which is very much a central paradigm with modern Jenkins (and thus obviously with Jenkins Essentials and Jenkins X.) See our very first page of the tutorial! https://jenkins.io/doc/pipeline/tour/hello-world/

I defer the question about container building in Kubernetes to somebody from the Jenkins X team, but I wanted to respond to your side note.

Offloading build queue outside Jenkins to another service, auto-scaling of build servers, configuring Jenkins with your configuration management tools are all something we are thinking about / looking into / actively working on. Some of them haven't gotten to the point of proper write-up yet, but see

- https://github.com/jenkinsci/jep/tree/master/jep/201

- https://github.com/jenkinsci/configuration-as-code-plugin

- https://github.com/kohsuke/jenkinsfile-runner

Creator of Jenkins & CTO of CloudBees here.

Thanks for using Jenkins for close to 10 years and sorry to see you move on, but I just want to correct the record here because I don't think the time line of events and your description are accurate.

First, pipeline stages have never been restartable in Jenkins from the beginning of Jenkins Pipeline. It wasn't as if we started with restartable stage and decided to close-source it at one point. From the very beginning, it was a feature we exclusively developed for CloudBees products.

From time to time, we do move some features from products to Jenkins. As somebody later in the thread pointed out, in JENKINS-45455 we are doing just that. Another example of this from early days is the folders feature, which is now used by many.

Any company building enterprise products on top of OSS will likely keep some features in products. And for any given person, only some of those features are likely useful. So while I understand the frustration of "that feature should be in OSS" or "I should be able to just get this one thing for a small price", I don't think there's anything inherently bad about these practices.

As for Pipeline stage view, it is still available today, and IIRC it is also still a part of Jenkins 2 default experience. Now, you are right that, as a contributor of the project, CloudBees is focused on pushing Blue Ocean forward. We think Blue Ocean solves the problem of pipeline result comprehension a lot better, and we'd rather make one solution better as opposed to work on two separate things that solves the same problem simultaneously. That is not to block other people from carrying the pipeline stage view forward, though, if anyone is willing.

I hope that helps,

I'm the creator of Jenkins.

Without fully knowing the context, I wonder if this is just a simple disconnect.

In Jenkins, we collectively cover wide-ranging use cases not by making one existing plugin (say Git plugin) do everything, but by making it extensible so that other people can define additional semantics and behaviours as separate plugins on top of it.

We learned this in the early days of Subversion plugin, back when that was the most popular version control system. Everyone uses a generic tool like that very differently, so as we kept adding individually valid use cases to the Subversion plugin, it became this giant hair ball not just for devs but also for users.

This mode also works better for those who do not want to spend time explaining why their use case is a good/important one, as they can simply code up their idea as a separate plugin and move on.

With all that said, I'm sorry for the frustration you had. Do you still have some pointers to tickets/PRs, etc? I'd like to look into it.

If I understand you correctly, you are talking about managing Jenkins configuration as code. For that, see

* configure jobs as code (https://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin)

* system configuration as code (https://issues.jenkins-ci.org/browse/JENKINS-31094) and its current prototype (https://github.com/jenkinsci/system-config-dsl-plugin)

* bypassing setup wizard (https://issues.jenkins-ci.org/browse/JENKINS-34035)

There are also puppet modules, Chef cookbooks, and etc that a lot of people use. Some of those are maintained by people who are also in the Jenkins community, so I don't consider them "3rd party hacks" if you are referring to those.

There's one word that I expected to be in the list but it isn't, which is もったいない (mottai nai)

It describes the situation where something/someone that is still of some use is left unused and wasted, such as leaving food on the table or wasting somebody's talent.

I'm a Japanese who's been living in the US for more than a decade, and I always thought this one word represents what's missing in American culture the most.

I just went to Tsukiji on the opening day of this new year. As the article states, this is the last year of the 70+ year history of the Tsukiji market.

If you can read it, this document <http://www.shijou.metro.tokyo.jp/pdf/book/book_all.pdf> by the city government, who owns the market, covers the context, motivations, and efforts that had gone into the migration of the fish market from Tsukiji to Toyosu.

I was originally skeptical why it couldn't be re-developed in its current site. After all, many more ambitious redevelopment projects have been successfully done in Tokyo. For example, Shibuya station, which is used by more than two million people every day, has been going through a major redevelopment. The same with freeways in Boston aka "Big Dig." Granted, these projects take long time and is a major annoyance, nonetheless it proves that massive redevelopment projects are possible.

The document I mentioned earlier gives a nice short overview of why it is not so for Tsukiji. The main arguments are:

* Redevelopment needs a temporary on-site space but Tsukiji is so overcrowded at this point that this is not possible

* Unlike users of train stations, users of Tsukiji (food, fish, trucks, and cars) aren't very compatible with multi story buildings, yet the site is too small to have a flat single story building, which is why they feel the need to move in the first place.

* The government wants to build a closed walled building for the new market. In contrast, today the market has just the roof and no walls, and it's indeed not very clean (at least for the Japanese standard.)

In reference to the soil contamination problem mentioned in the economist article, the government apparently replaced the top 15 feet worth of soil for 40ha, which had cost whopping half a billion dollars (!)

I'd be curious to read a similar comprehensive argument from those who oppose the migration.

I'm from the Jenkins project.

I wish the authors of this post gave us a heads up beforehand. It put our users at unnecessary risk.

At Jenkins project, We've published a mitigation script (https://jenkins-ci.org/content/mitigating-unauthenticated-re...) while we work out a better fix for users.

I always thought a better way to combine programming & minecraft for kids would be something like ComputerCraft (http://www.computercraft.info/), where you can program a robot inside the Minecraft world to do stuff for you, like building road/bridge, digging tunnels, etc.

ComputerCraft is Lua-based, but I can imagine something similar based on CLR, and that way you can let kids program in C# from Visual Studio.

Programming a bot is much simpler and even a very simple program (like digging a tunnel straight) is instantly gratifying. Writing a mod has a much higher barrier to the entry, AIU.

Personally, I don't feel all that strongly that we are collectively getting it wrong, certainly not to the extent of getting a domain name just for that! I think most developers do indeed talk to their users, and they rather enjoy doing it, too. I've once worked in a place where developers are more detached from users, but that was an exception, not a norm.

There's also a difference between the opinion of one user vs the collective opinions of users as a whole. In most software these are conflicting goals. For example, iOS can't just add random features willy-nilly to satisfy an individual's feedback. So the product management needs to come in between developers and users to aggregate user demands. You'll have to make everyone a little bit unhappy to make users collectively happier. One of the reasons I like extensible software is that those two goals are no longer conflicting. Hopefully you know what I mean when you see Jenkins (http://jenkins-ci.org/)

I do share the joy of "caring about your users," there's something very special in knowing this one person/user, understanding his/her needs, solving it, and making him/her happy. We obviously do that through software, but this drive is universal.

Being from Japan, I have a few thoughts in this area:

* A significant percentage of music is still only available on CD, and not in digital media. Until recently, music from one company (say Sony) was only available from their digital music store, which had their own player app with their own DRM. These things create significant inconvenience to the consumers, hindering the adoption of digital music.

* The music industry in Japan came up with a brilliant business model that enabled them to sell the same CD multiple times to the same consumer, basically by bundling a ticket to each CD. Sometimes a single ticket grants you one vote, sometimes you get one piece of (say 12) tickets randomly and you need to complete all 12 pieces to get something. This forces loyal fans to buy 10s and sometimes 100s of the same CD.

It's not like people are carrying around portable CD players any more, so these factors obviously creates a rapidly diminishing music fan base. I still hope that eventually the decline in the CD business will force the industry to wholy embrace the digial music, but I'm not holding my breath.