scottdw2

This seems to be, quite obviously, an attempt at free marketing. They want to be able to point at the corpus of products influenced by SO, and say "see, we're pretty cool".

Seems misguided to me.

There are several connections between Lambda and functional programming.

1. The snippets of code you run in Lambda are called "cloud functions".

2. Lambda applications are generally event driven and stateless, which is a very functional idiom.

As other folks mentioned, however, you can use Lambda with non functional languages (like Java). So even though a Lambda application is inherently functional (small stateless functions communicating via events), you don't have to write in a functional style if you don't want to.

Your code looks correct. I would expect something closer to 50ms in the warm path (300ms in the cold path seems about right).

I'll take a look tomorrow and see if I can reproduce what you are seeing. I'm not super familiar with API gateway, so there could be some config issues over there.

If you want to discuss this more offline, feel free to contact me at "scottwis AT amazon".

Lambda has a fairly generous free tier.

Here are the details on pricing:

https://aws.amazon.com/lambda/pricing/

You get up to 1M requests / month and 400,000 GB-seconds of compute time per month.

A default lambda function uses 128 MB of ram (0.125 GB), which gives you about 3.2 M seconds of compute time (time actually spent executing requests) every month for free.

Thus if you have functions that take 500 ms on average, and use the default amount of RAM, you can process about 6.4 M requests in a month for a total bill of $1.20.

Above the free tier limits you pay $0.20 per million requests, and $0.1667 per million GB-seconds.

The pricing is fairly attractive.

Are you using Node.js or Java for your Lambda function?

If you are using "Node.js", you may be seeing slow times if you are not calling "context.done" in the correct place, or if you have code paths that don't call it.

Not calling context.done could either cause Node.js to exit, either because the node event loop is empty, or because the code times out, and Lambda kills it.

When node exits, the container shuts down, which means Lambda can't re-use the container for the next invoke and needs to create a new one. The "cold-start" path is much slower than the "warm-start" path. When Lambda is able to re-use containers, invokes will be much faster than when it can't.

Also, how are you initializing your connection to DDB? Is it happening inside your Lambda function? If you either move it to the module initializer (for Node), or to a static constructor (for Java), you may also see speed improvements.

If you initiate a connection to DDB inside the Lambda function, that code will run on every invoke. However, if you create it outside the Lambda function (again in the Module initializer or in a static constructor) then that code will only run once, when the container is spun up. Subsequent invokes that use the same container will be able to re-use the HTTP connection to DDB, which will improve invoke times.

Also, you may want to consider increasing the amount of RAM allocated to your Lambda function. The "memory size" option is badly named, and it controls more than just the maximum ram you are allowed to use. It also controls the proportion of CPU that your container is allowed to use. Increasing the memory size will result in a corresponding (linear) increase in CPU power.

One final thing to keep in mind when scaling a Lambda function is that Lambda mainly throttles on the number of concurrent requests, not on the transactions per second (TPS). By default Lambda will allow up to 100 concurrent requests.

If you want a maximum limit greater than that you do have to call us, but we can set the limit fairly high for you.

The scaling is still dynamic, even if we have to raise your upper limit. Lambda will spin up and spin down servers for you, as you invoke, depending on actual traffic.

The default limit of 100 is mainly meant as safety limit. For example, unbounded recursion is a mistake we see frequently. Having default throttles in place is good protection, both for us and for you. We generally want to make sure that you really want to use a large number of servers before we go ahead and allocate them to you.

For example, a lot of folks use Lambda with S3 for generating thumbnail images, sometimes in several different sizes. A common mistake some folks make when implementing this for the first time is to write back to the same s3 bucket they are triggering off of, without filtering the generated files from the trigger. The end result is an exponential explosion of Lambda requests. Having a safety limit in place helps with that.

In any case, if you are having trouble getting Lambda to scale, I'm happy to try and help.

In a leader election, a node won't vote for a candidate that doesn't have a record it believes to be committed. Thus, writing to a majority ensures that when the leader dies, the new leader will have all the committed dats. Thus, up to n/2 - 1 nodes can be lost with a guarantee that no committed data is lost.

If you don't need strong consistency (after a write commits, all future reads will see the write), you can use simpler replication strategies.

Redis, for example, using async replication, that is not guaranteed to succeed. A redis master may ack a write, and a subsequent read may see it, but a loss of the master before replication occurs can result in data loss. The failover is not guaranteed to contain all writes. Some times weak consistency is ok. Sometimes it's not, but eventual consistency is ok. Other times strong consistency is needed.

Raft is useful when you need strong consistency.

Raft does support membership changes (adding and removing nodes from the cluster), so it can support losing more than n / 2 - 1 nodes, just not simultaneously.

A few things:

1. You don't have to round trip to every member before acking a write request, only to a majority of them.

2. If you use the right data structures for your state machine (like persistent data structures), you don't have to fully commit a write before you start the next write. That is, as long as you can quickly revert to a previously committed state (which you can with persistent data structures, it just becomes a CAS on a pointer) you can pipeline requests and batch commits. That lets you start on write n+1 using the state machine results of write n, even though write n hasn't been committed yet. You can then commit a bunch of results (n, n-1, n-2, ...) as a single batch. That does create a dependency that some write j may be rolled-back because a write i < j failed to commit, but you would have had that dependency anyways (you wouldn't have even started write i unless write j had committed). So, although latency of a single request is bounded by the time to replicate a log entry to n/2 + 1 hosts, the throughput is not bounded by the latency of a single write.

The raft paper doesn't actually say you can do this (it says things like "don't apply state machines until a transaction has been committed), but that's because it assumes arbitrary side effecting state machines. However, with some restrictions (like using persistent data structures), you can relax that requirement. Relaxing that requirement can give quite good throughput.

3. You can horizontally scale raft the same way you horizontally scale any other distributed data store: using consistent hashing. You would use a hash ring of raft clusters.

4. You can loose up to n/2 hosts from a raft cluster and it will still work.

5. Raft is "multiple servers with failover and a version clock". In fact it's "multiple servers with failover, a version clock, and strong sequential consistency".

This is a nitpick, but it is lazy.

It's not represented in head normal form, but it's still lazy.

The limitations you mentioned are not an artifact of the implementation being eager(because it's not eager), but instead are an artifact of the representation used.

Something like that may work for digital goods.

You deliver some digital assets, the key to which can only be derived if a particular crypto currency transaction is performed.

But if you are delivering physical goods, there's going to be some human aspect to it.

Some human, somewhere, is going to deliver the good to some other human.

I'm not discounting the idea of automated contract enforcement. I think contract enforcement is really inefficient. Finding a way to automate it seems like it could be useful. That's the niche paypal originally filled (providing escrow services for ebay transactions). If you could find a way to automate such things, so that everywhere you currently see a "standard service" contract nowadays you had someone clicking on a button (and maybe entering a credit card number), it seems like it could be disruptive. Kind of like a Google App engine for executing contract logic.

I'm just not sure that bit coin is central to the whole thing.

The whole "free from government interference" thing is a bit of a fallacy. If the government wants to seize your bit coins, they will find a way. It may be difficult for them to manipulate the price of bit coin, and to implement wholesale theft (stealing everyone's bit coins), but seizure of a specific individual's bit coins is definitely possible. They'll find a way.

Maybe some form of contracts, like options trading, might work. Equity funding of startups might work too. Basically anything that's purely digital.

But I don't think that any contract involving physical goods can be rendered mathematically perfect.

There's an interesting split here: between contract validation, contract enforcement.

Both have typically been handled by courts.

Contract validation is typically very "human centric". It centers on questions such as "what is a bargain?" or "is this result acceptable to the conscience (I.e. is morally acceptable)".

Contract enforcement, however, is much more formulaic. Given a valid contract, and a series of events (leading to a breach), quantify what relief would look like.

I don't think validity can be automated. It's analogous to Dijkstra's obsession with "formal verification".

However, most contracts are valid. They typically involve genuine negotiations on reasonable terms.

Assuming validation, enforcement is easily automated.

I don't think bit coin is relevant though. For automated contract enforcement any payment API will work. Wether it's bit coin, or paypal in Estonian Kroon, the important part is automation.

There are already 1M apps in the App Store.

There's already too many apps, so preventing "over choice" isn't their goal.

A shelf space analogy doesn't make sense. With a real shelf, there is scarcity. Only so many shelves can fit in the store, and only so many products on a shelf.

The marginal cost of a virtual good is zero. Apple's ability to support apps is effectively infinite.

There are over 1M apps in the App Store.

This is not about "shelf space".

In war, identity is a defined "negatively". That is, not by describing positive attributes to you, but by describing negative attributes to the enemy. "We are the opposite of x", hence we fight. If you neutralize the ability of the enemy to define you (because he is like you) he can't define himself, and so he cannot fight you. Thus, there is no enemy, and hence no war.

Is there any better way to change someone's mind, to change their fundamental mode of thought, to alter their perceptions and manner of reason than to educate them?

Here's an even better question: if your enemy thinks like you, talks like you, acts like you, shares your values and views the world the way you do, is he your enemy? Can he be your enemy? Why would he be your enemy?

Let's consider a central question.You espouse a belief in freedom, a nation built on liberty. Your enemy denies this, proclaims you a tyrant, and endeavors to make war against you. How do you respond?

One method undercuts your enemy's message, reaffirms your core values, and has the potential to alter and modify your enemy so that he is predisposed to be united with you.

The other affirms his message, strengths his position, and emboldens those who follow him.

This leads to the last question: what is your objective? Is it victory, or something else?

What better path to victory then to alter the mind of your enemy until he is incapable of fighting you?

Our educational assets ought to therefore be shared. To sensor them in the name of security is foolish.

That's the whole point.

It's disruptive technology: not as good, way more accessible.

This bothers me.

I know this is a bit of a 'libertarian fantasy', but I think the constitution ought to be amended to contain something like the following:

1) No person shall be subject to any criminal penalty exceeding one year of incarceration or $300 in fines for any non violent offense relating solely to the possession, sales, distribution, manufacture, or purchase of an intoxicating substance.

2) The purpose of this article is to limit the scope of criminal penalties that may be applied to "non violent drug offenders". It's provisions shall be interpreted with such intent in mind.

3) This article applies to all jurisdictions with the Several States, the United States, and any territories or possessions thereof

4) Any forfeiture of assets resulting from the conviction of a "non violent drug crime" must be limited to:

a) The intoxicating substances constituting the "core element of the crime"

b) Any asset materially and predominantly used for the manufacture, production, and possession of such substances.

Provided that such seized assets do not also have reasonable, fundamental, predominant, and legally authorized uses. In such case any seizure must be subject to the provisions of "eminent domain".

5) Congress, or the states, acting within the provisions otherwise authorized by this constitution, may adopt measures to ensure assets actually used in the commission of a "non violent drug crime", when not seized in accordance with this constitution, are only used in accordance to lawfully authorized purchases.

The key take away from the article SHOULD be: don't rely on ntp if you don't have to.

There are people who have to. They run their own atomic clocks, and worry about things such as precision delivery of nuclear ordanance.

Then there's you. You should use vector clocks, with a builtin conflict resolution mechanism based on domain knowledge.

That's the point of the article.

I won't argue that constraints can lead to innovation. All art exists in some medium. Constraints are therefore inherit.

Here's a simple question: what spurred Picaso's decision to paint in Blue? Was it adherence to a protocol describing blue as the answer to his expressive woes, or was it his own internal experimentation with the color blue?

Did Picaso paint in blue because he felt like Blue, or because someone said "any work 'submitted' to this gallery must be blue"?

Sigh. Not about the original article, but about the top comment in the original article.

It presumes that Picaso, Bansky, and Monet can't paint like each other.

Probably not true if you actually have Picaso, Bansky and Monet.

In any case, style guides are for hacks, not artists.

Actually he has admitted he was wrong about the iPhone, thinking of it as a high end phone instead of a low end (but very convienient) computer.

The author of the article, I think, makes a "phalous leap" from "He was wrong about the iPhone" to "disruption doesn't apply to consumer markets".

Primarily because the iPhone is a perfect case of this. It (and it's successor, the iPad) is disrupting the PC industry.

You have to look at the phone as a computer, not a phone. Then disruption theory REALLY makes sense.

The most interesting part is that the difference in votes is exactly equal to the number of individuals marked as "not voting".

Actually, no they couldn't. Patents only apply to the right of "first sale". You can resell any patented invention without the consent of the patent holder.

Jason has a point:

You can maximize revenue by focusing on delivering higher value features to more lucrative customers.

But, you need to be careful when following his advice. Blindly following the principals of "revenue maximizing resource allocation" makes you susceptible to disruption.

If you focus too much on high-end customers you can either:

1. Make the product in-accessible to a large audience.

2. End up overserving many of your customers.

That opens up the opportunity for new entrants to provide less-good products at the lower end of the market, which you would be happy to ignore (you can just write a $80 book next time, targeted at fewer folks), while the new entrant follows you up market, until you have no business left.

It's not clear it applies here, as neither author has a structural advantage over the other. However, just because you can earn more profits by charging a higher fee to fewer customers, doesn't mean you should. It could kill your business in the long run.

Take Windows PCs vs the iPad as a good example.

If there was a sudden dramatic rise in the number of cases approaching trial, the natural response would be to increase the value of plea deals until the number of trials drops to a managable level.

As the supply of plea deals increases, and the demand decreases, their price will drop.

Prosecutors with a large volume of unsold plea deals (a large inventory of pending cases and intense pressure to clear them quickly) will be highly motivated to reduce the sentences offered to criminals.

The net result would likely be smaller sentences, but not a fundamental reform of the justice system.

This is basically a form of strike. Although strikes can improve the wages of workers, they won't fundamentally reform the relationship between a company and it's workers.

No strike by the UAW would ever lead to the creation of something like YCombinator, for example.

The key to fixing problems with our government is to disrupt it. To replace it with something that is "not as good", but "much more accessible" (if you are Exon, the current government is great, because it serves your needs exactly, but if you are a non-violent drug offender it is completely in-accessible to you).

See my comment below. I don't think you quite understood what I meant. I'm not saying the code was written in scheme. I'm saying there is a product that allows you to write scheme macros to manipulate a database of machine code IR derived from disassembly and then turn the modified database back into an executable.

Hiding the source language makes identifying the origin of the malware difficult. There are obvious reasons to do that.

No... the product allows you to write scripts to manipulate its machine code IR database in scheme, and then spit out the machine code as nasm assembly, assembly them, and then run the appropriate linker in the same way that was used to produce the original exe. Scheme is used as a macro language. So you use scheme to say: change the code at EA 0xdeadbeef from a mov to a jmp. You can reorder functions, insert and remove code, etc. It works because it has very high quality disassembly based on observing compiler and linker invocations and introspecting the artifacts involved.

The payload could have been modified (to obfuscate its origin / source language) using a product named codesurfer/x86.

http://www.grammatech.com/research/products/CodeSurferx86.ht...

If it has access to source code, it can instrument the build process, and obtain disassembly that is high quality enough to support rewriting. Using it's scheme API you can modify the CFG of each procedure directly, serialize the rewritten parts out as nasm, and even relink with the object files you don't have source for.

It works with any build system, and supports gcc / as / ld and cl / link.

So it may not have actually been written using a custom pl.

The US constitution also grants a right to a jury trial in "all suits at common law where the value in controversy shall exceed twenty dollars".

All US judicial authority derives from the constitution. Given a controversy under the jurisdiction of the Federal courts, a trial cannot be denied the agreeved party unless:

1. There is no controversy

2. There is a stipulation in the constitution, or the jurisdictional laws passed by congress, that says federal jurisdiction doesn't exist.

3. There is a valid treaty which specifies a different process

4. The case was already tried.

5. Neither party is subject to the laws of the US.

Absent those conditions the US courts would have to take the case.

Forum selection conventions must still conform to the constitution.

If you wanted to change this you would need to convince Congress to pass a law changing enforcement of copyright.

No.

Government is force. Airlines, like most businesses, are subject to market forces. If Delta said "we have to search the data on your computer before you can fly", you can choose not to fly. Someone would have a strong incentive to provide a mode of travel that did not involve laptop searches.

I didn't say "no government". I said "small government". There's a difference. You need the force of government to protect property rights, and to enforce contracts, for example.

I don't think you understand. The point is to either eliminate the power, or make it extremely diffuse. That way, there is nothing to purchase.