HN user

ideaoverload

148 karma
Posts14
Comments39
View on HN

I am senior developer/data engineer based in Poland. 2021-2022 I was getting remote work offers from all over Europe, some from quite interesting places. Today I am still getting offers but mostly from Poland, lower rates or not disclosed, more boring 'build an ETL' type. There is a definite change but looks like it would still be possible to get a new job quickly, possibly at lower rate.

Polish example seems to be some mix up of forms:

  Jestem z Polski == I am from Poland
  Jesteś z Polski == You are from Poland
  Jestem Polakiem == I am Polish
  Jesteś Polakiem == You are Polish  
Both have very close meaning but 3&4 seem to be equivalent of examples in other languages given above.

Source: I am Polish

I am tasked with standardizing ETL in a small org, airbyte is on my list to evaluate. As I write this there are 64 comments in the thread and only a single comment mentions actual experience using airbyte. Any other actual insights using the tool? What about Airflow downsides mentioned in the article? Thx!

Ask HN: What are the best hacks to fight <insert illness that can kill you or ruin your life>.

Start from trusted resource like: https://www.nhs.uk/conditions/stress-anxiety-depression/low-...

It is possible that you will need to work with therapist who has spent decades debugging and solving problems like yours. If you can't afford therapy try to talk to real people with years of life experience outside of hacker forum.

At the end of the day you will need to understand what is the cause of your problem. Start thinking about it right away.

Same here. I kept notebook full of solutions I have found. My favourite trick was to use sets of 2 pieces placed next to each other that could be flipped without need to rearange other pieces. This way I could get 2 slightly different solutions in one shot.

3 year girl in my family had serious gastrointestinal issues. She suffered on the daily basis and it made contact and therapy just much much harder. Diet adjustment was huge win. It does not cure but enables development.

There are many publications that suggest that autism spectrum disorders are caused by different neuron wiring compared to so called neuro-typical human. The nature of this differences is not well understood but it does not sound like something that can simply 'treated'

EDIT: good article how some people are able to function with autism (which may look like treatment but it is not) http://www.nytimes.com/2014/08/03/magazine/the-kids-who-beat...

I was working on such system for performance measurements. Basic assumption was that user would never see any mention of tables, joins, column names nor any other SQL concept.

1.Table selection - there is predefined set of joined tables we call views. Views have descriptive names e.g. 'Temperature measurements', no underlying tables are shown.

2.Column selection - user selects columns from views using descriptive names , not actual column names . UI clearly indicates columns that land in 'group by' section by calling them dimensions. There is predefined aggregation strategy for all columns that are not dimensions - e.g. average for temperatures or sum for number of measurements. In some cases more that one aggregate is available e.g. average or maximum temperature. User just selects maximum or average temperature not aggregation operator.

3. Filters: user can select simple filters on columns: =<> and string matching for text. All filters are ANDed

4.Sorting: user can select columns to sort on.

5.Display: results are displayed as tables or charts. More that one section may be placed on single report.

The real system has tons of additional features but basic design as design as described above has worked great for years.

Why does this happen even when the instance isn't using all of its CPU?

Probably hypervisor does not assign CPU every time it is requested but it still manages to assign as much as needed because in the end there is some idle time left.

Description of noisy neighbor problem #6 lacks some depth.

AWS noisy neighbors problem is very often misunderstood. CPU steal time under linux does NOT mean that somebody is stealing your CPU. It simply means that you wanted to use CPU and hypervisor has given it to another instance. This may happen because you have exceeded your quota or scheduling algorithm selected another pending instance at this very moment and it would give CPU back to you a bit later. In the end in both cases your instance gets fair share.

Great detailed explanations of steal time: https://support.cloud.engineyard.com/entries/22806937-Explan... and: http://www.stackdriver.com/understanding-cpu-steal-experimen.... The latter article is mentioned by OP but seems to be not fully read/understood.

Why does killing and restarting instance help? It likely moves instance to different hardware node with less active neighbors. When your neighbor is not active your instance can use CPU idle cycles of your neighbor! You sort of become the noisy one. Still hypervisor would prevent it once neighbor starts to fully utilize his CPU quota and you are back to square one.

Amazon does not oversubscribe CPU according to their CTO: http://itknowledgeexchange.techtarget.com/cloud-computing/am...

Amazon specifically states that t1.micro instances do not guarantee CPU performance: "Micro instances are a very low-cost instance option, providing a small amount of CPU resources. Micro instances may opportunistically increase CPU capacity in short bursts when additional cycles are available. They are well suited for lower throughput applications and websites that require additional compute cycles periodically, but are not appropriate for applications that require sustained CPU performance."

While CPU sharing is pretty well documented noisy neighbor problem still exists for network and disk resources being shared by multiple instances on the same hardware node. The only way to detect these problems is to track network throughput/loss rate for network and IO stats for disk.

You are guaranteed to avoid noisy neighbors CPU problem by using AWS dedicated instances: http://aws.amazon.com/dedicated-instances/

I work for APM ( Application Performance Management) vendor , I have no business praising AWS.

[Edit:spelling and clarity]

[Edit2: changed CPU scheduling description]

I like the idea. While I understand you need a decent sample size but I am not sure I will patient enough to login 29 times. Some feedback 1.it is not possible to cancel test execution 2.it is not clear if I need to complete all tests each time to get a result at the end

In Europe you can buy a prepaid phone card and use your mobile phone/modem to get online anonymously. Phones can be identified by IMEI id but you can physically hide the device or likely swap IMEI when needed using software. Not sure if other ways to remotely identify phone/modem exist.

API serves as a way to file lawsuit. It is humans who process them. This court deals with standard claims like businesses not paying for the service/goods but also consumers not paying their bills.

API is useful for companies like mobile operators or utility providers who have thousands of subscribers who do not pay.

1.6M payment orders, 21 months, 200 staff = ~17 cases per day per person (no holidays and vacations). Sounds doable given it is likely to be batches of claims as described above.

Time.is 14 years ago

The clock display is nice but not a killer for me. Here&There time comparison in different time zones is the best I have seen. Full page calendar with multiple months is very simple but works great for me. This is becomes my favorite site for time/date related stuff.