ideal0227

Oollama.ai focuses on making it as easy as possible to run models locally. We aim to provide a seamless experience that feels the same whether you're developing locally or deploying remotely for production.

declaimer: work at Lepton AI.

No, Paxos is not a shard log abstraction. Paxos is about agreeing on one value rather than a set of values in sequence. So my possibility 1 is true.

What is not true? 10x simpler, no?

There are two possibilities:

1. You misunderstood Paxos, probably confused that with RSM. For example, Paxos is just the leader election part of Raft. Superficially, it seems different from Paxos, but it what it is under the surface.

2. You implemented RSM incorrectly, probably missed some important features or optimizations. For example, log compaction, membership reconfiguration, pipeline, back pressure on execution, etc.

It is VERY important to differentiate Paxos and RSM. There are tons of optimizations you can do with RSM. But on consensus, there is ONLY Paxos today. Or you do it wrong or you invite something truly new.

There are three main stages about RSM.

1. log replication (m) 2. log consistency (n) 3. log execution (k)

Then you will have m * n * k ways of achieving your goal based on different requirements on the three stages.

I think all these kinds of papers are very confusing. Comparing RSM (replicated state machine) to Paxos is just like comparing a car to an engine. It makes very little or no sense.

In the original Paxos paper (https://lamport.azurewebsites.net/pubs/paxos-simple.pdf), the part 3 (RSM) is not extensively explained. There are countless ways to use Paxos to implement RSM. Multipaxos/Raft/Epaxos try to fill in that gap.

By any means, Paxos itself is 10x simpler than Raft or whatever. Every time I heard a "distributed system" engineer said Paxos is complicated, I know he/she does not have much experience in the field or at least has never implemented the core consensus part...

This is solved by throwing multiple interview questions at the person that are all novel and not derived from the internet. The probability of ALL questions being seened before is very low.

This is solved by adding more questions into Leetcode. 1000+ and counting.

So? Doesn't change the effectiveness of leetcode in passing an interview and displaying your ability to learn and solve novel problems. If you have other ways of passing those interviews outside of leetcode, that's great. Use it.

You missed the point entirely. My argument is that many interviews are not designed for problem solving but practicing and memorization. And this is exactly why Leetcode ensures you to repeat these template 100 times effectively.

I do not plan to pass this kinds of interview, before, now or the future. So why do I bother?

Good luck on your job search anyway. I have no intention on discouraging you to do the practice. I cannot fix anything here.

Google specifically tells their interviewers to avoid "canned" problems that are already all over the internet.

They did a pretty poor job on this already. And it adds one more requirements, the ability to pretend that you have never practiced a similar question before :P.

Leetcode isn't about memorization. Leetcode is training data used to optimize your neural net to solve problems it hasn't seen before.

Oh. Thanks. Glad that Human brain does not need that much data like today's neural net.

And there are millions of ways to improve problem solving than repeating BSF/DFS/DP templates on Leetcode questions.

Thanks for educating me about BFS.

I am major in computer science, and published paper in top conference. Tree traversal is trivial for me. So I guess I have decent CS background and knowledge?

Writing bug free and perfect code is not equal to simply solve the problem. If you have prepared coding interviews (especially for new grad and internship), you should understand it is less about knowledge but more about practicing and memorization for that specific purpose. The time spent on it is almost a waste in the future.

If the interview is knowledge based or problem solving oriented, I am all for it. Sadly, it is not today in many places. And that is exactly why website like Leetcode exists.

When my classmates were preparing interview coding questions, I was working on a mini TCP implementation and a toy kernel.

AWS rejected me since I failed to write prefect code to traverse a tree in level order. Google did not even give me an interview since I told the campus recruiter I have not prepared for the coding questions.

Then I ended up with an internship at CoreOS and created etcd. I am glad that they did not hire me back then.

Today, I am sure I still cannot pass the coding interview at "Giant Search and Advertising Company", but they run a lot of my code in production :P.

To improve read perf, you can add learner members or add a layer of cache proxy.

It uses etcd/raft from beginning.

Let me share a little bit more story here.

The original go-raft is one of the first raft implementations. At that time, the raft paper was not even officially published. Many other attempts around that time were not very successful, including go-raft.

Making a production ready consensus algorithm is not easy: https://www.cs.utexas.edu/users/lorenzo/corsi/cs380d/papers/.... Things like pipelining, batching, flow control, asynchronous snapshot were not extensively explored in the context of raft. And not much effort has been put into testing due to the immaturity of the applications of raft at the time.

We realized the problem a few months after etcd alpha was initially released and became popular. However, I went back to CMU to continue my master degree for 1yr, which slowed down the progress.

After I came back from school, together with Blake, Yicheng from CoreOS and later on Ben from Cockroach Labs, we built a solid raft impl as our first priority. Once we put etcd/raft inside etcd2, the stability of etcd greatly improved. That is about 1.5 yr after the initial release. Now etcd/raft powers many production level distributed systems: tikv, cockroachdb, dgraph and many others.

Over the last couples of years, the focus of etcd/raft is always stability and nothing else (although people are blaming us for usability :P).

No. You are wrong. etcdv3 enables l-read by default.

That is true. etcd v3 provides l-read by default. You have to opt-out that to improve latency.

If you never work on Unix, then you should insist on saying so. Holding integrity is much more important than getting an internship from FB. And, as an engineer working in the infrastructure space, I do not believe FB is looking for someone has UNIX experience for the internship.

The recruiter is not knowledgeable. If he/she is hiring for production engineering, he/she needs to understand the basic of the keywords they search for. No reason for you to apologize.

Reading your opinions in this thread is really like reading jokes.

No facts are provided to support your opinions, and all the suggestions are ridiculous.

There are two core issues:

1. Liu is not acceptable by a lot of Chinese people due to his aggressive words on colonial and slavers.

2. Liu was sentenced by the gov due to "subversion".

So to judge Liu, we have to look from at least two different angles, not just "he was sentenced".

Few people thinks 2 is right. I, personally, do not agree on 2 too.

And I am now thinking why most people outside China only know about 2 when they try to judge about Liu?

Right. And you still know something is not acceptable by the people, and you probably wont go across that line. Same thing in China. Many of Liu's opinions are hated by Chinese people in general, not by the central power.

eh...

Liu was not sentenced due to this <三百年殖民地。香港一百年殖民地变成今天这样，中国那么大，当然需要三百年殖民地，才会变成今天香港这样，三百年够不够，我还有怀疑。>

And, yes, what you just described is also well known. No one in this thread said Liu should be sentenced. And no one in this thread said they agree with what Chinese gov did.

What we suggest is that some of Liu's opinions are not acceptable by Chinese people. And he is not well respected as people might think. Some facts are provided in this thread.

Since that is a well known issue about him. No one thinks it is conspiracy theory. If you want to discuss about Liu, I assume you should do some homework first?

There are a lot of problems with Chinese authorities. There are a lot of things I cannot agree on with them, and I am one of the Chinese people that are trying hard to fix them by doing real work.

Moreover, China != Chinese government. I, personally, hate the opinion that Chinese people who has a "traditional" view is inferior. Yes, they might be "washed" by todays "Chinese system" or education, and you disagree with them. But, it is not their fault.

And most of Liu's view is going towards the direction I really hate. Another example is his calling Chinese people "slaves transformed by the education system".

Again, I am not saying the authorities did the right or best thing. But just Liu is not the person that I would respect.

I can read Chinese as well

I am a native Chinese, which also implies I understand more of the language context. Also it implies I probably can share more of the sentiment as other Chinese people do. So, yes, there is a difference between how people view the exact same content. And I was trying to clarify that.

China would have to be colonized for 300 years to reach the same state as Hong Kong

Why China needs to reach the same state as Hongkong through Colonial? Does he feel HongKong was at a worse state compared to China at that point? Or HongKong was at a better state?

And, yes, no matter what state Hongkong was at due to the Colonial, China SHOULD NEVER go that path. That is where I cannot agree with Liu.

Any one can justify his words. But I will still hold my opinion on this specific issue after I read quite a lot about Liu, about what he said, about what he wrote, in both Chinese and English articles and documentary.

I don't care one way or the other, it's the Nobel committee's decision who they award the prize to. It makes no difference to me. It doesn't make what he, or any other Nobel Laureate say any more true or worthy of consideration.

Yea. That is what I want to learn. You do not care about the Prize.

If you want to argue in favor of why some views should never be publicly stated and who is gonna decide what is/is not allowed, go ahead.

It seems that is you want to know about it. Probably I misunderstood you. I have no interest :P. So we can stop here.

What are you trying to say? Is this a question related to my last view or trying to question me?! I cannot figure your point really.

三百年殖民地。香港一百年殖民地变成今天这样，中国那么大，当然需要三百年殖民地，才会变成今天香港这样，三百年够不够，我还有怀疑。

I am a decently educated Chinese native speaker. The quoted Chinese sentence from Liu is not acceptable to me. I do not think it is a direct comparison against HongKong.

I am fine with him arguing about changing the political situation of China or whatsoever. And, actually, the discussion on that should be more open. However, I cannot agree on his view about changing China through colonial no matter what he tries to justify. And I believe no rational Chinese people will agree him on this.

Even if it is repressive, preventing him to say it by arresting him and his family is not repressive to you?

Are you sure he was arrest because of saying that? Do you have any facts to support your view?

Liu said China needs to be colonized for 300 years to get better, is it just free of speech? sound quite repressive to me.

Do you think Liu deserves Nobel Peace Prize after saying that publicly?

Remember that the argument was "There is certain views that should never ever be publicly stated".

No, it is not mentally similar I am afard. Implementing Paxos is a joy since it is a process of learning new stuff. Preparing for the coding interview is wasting time, which can put onto something much more meaningful.

I wrote a raft implementation that is used by quite a few well known systems (etcd, cockroachdb, docker swarmkit, tikv and quite a few closed source systems). Probably it qualifies the "production ready" standard :P.

With that experience, I can say that a real world raft implementation is not easy and VERY time consuming.

However, preparing the coding interview is definitely a order of magnitude more difficult for me. It requires me to waste all my time on something meaningless, and makes me feel sick.

There is some overlapping. But we should choose solution wisely :P.

Here is a doc [https://github.com/coreos/etcd/blob/master/Documentation/lea...] comparing etcd with other systems, including CockroachDB.

I work on etcd.

Well... Why do you need 2PC if nothing is sacrificed? :)

If you choose things like Raft (or other RSM), you can get a consistent key space to do txn over keys.

For a metadata store like etcd, one RSM group is OK. Performance/availability is better than something requires 2PC.

For database systems, there are quite a few txns that can actually fall into one shard and can bypass 2PC.

Of course, it is highly depend on the workload and use cases.