HN user
@nabla9 have tried to tell you that for DGX Spark, you can also use optimized models; therefore, this means that Spark can also be used for inference with bigger models, such as those exceeding 200B.
Please compare the same things: carrots VS carrots, not apples VS eggs.
You can look at the /DataHoarder subreddit, as I see people usually buy secondhand LTO4-LTO7 tape drives and libraries, which are affordable. Datahoarders use dozens and hundreds of cartridges for archiving/backup purposes or for making fun playing with tape drives.
The weather app doesn't give much money. The main business sells weather and climate data B2B: agro, insurance, logistics, retail, supply chains, advertisement, medical, etc.
Companies whose primary business is weather apps are small, and such areas are highly competitive.
why NOAA in the US is under attack
As far as I know, AccuWeather is the main beneficiary. You can easily find reliable sources about it.
The cause is that NOAA publishes all weather data, calculated models (global coverage), meteostations data (global coverage), and weather radars to the public for free (US only, maybe also Canada, I don't remember). Therefore, many weather companies use such data to do their business and compete directly with AccuWeather. They don't like this.
On the other hand, state weather agencies that calculate global models in many countries don't provide such data for free. Therefore, startups and small companies who work in weather and climate fields use NOAA data and directly compete with AccuWeather or don't pay them for data access.
I don't offer any solution for the problems you mentioned, and I don't think it is possible.
If we want to have global money and a global payment system, they would be controlled by governments, international organizations, God, Devil, Cthulhu, Spaghetti Monster...
There is no magical solution. We, as a society, need to establish competing social institutions, and try to control them, and try to force them to compete. There is no solver bullet.
Don’t lie to yourself, bro.
If we want to use technology in real world to solve real world issues, we need to consider all important non-technical things.
Nuclear Energy is great, but governments and international organizations want to control it because it is too dangerous. So, if we need to use Nuclear Energy, we must play by such rules.
Money is the same thing. Each government wants to control them, regardless of their form.
If someone wants cryptocurrencies to be widely adopted, there is no option but to give them to businesses and governments.
So, crypto would be regulated like usual money. Major blockchains have records for all transactions, which can be tracked and used by businesses and governments to implement more strict control over the whole world. Therefore, the more people use crypto, the less privacy they have.
The Internet and Web were designed to be anonymous, but cookies, IP addresses, data collection, ML/AI, IMEI, MAC, and the control of registration in ISPs and mobile operators have led us to a situation where the government and companies can easily track people. The same situation would be with crypto, which was designed to be anonymous but used in another way.
Don’t lie to yourself, bro.
You have bizarre logic here. For example, in a topic discussed about GPUs, someone would say that it's not possible to run databases on GPUs, so GPUs don't have any chance of succeeding.
How much do you trust your government with your money?
Do you trust crypto companies? Mt. Gox, FTX, Bybit…
Do you know that crypto companies must follow government rules, regulations, and laws? Russians were banned from using many crypto exchange platforms. China has strict rules for its citizens. You can buy and sell crypto in Brazil, but you must use only Brazilian reals.
Pix isn't global, but no one government person outside of Brazil can block this system.
MasterCard, Visa, Amex, and UnionPay work worldwide, but only a few countries regulate them, can block their usage, and can use data for tracking and statistics.
Pix is free to use, so no one needs to pay an additional "tax" to MasterCard and Visa (it's about 3%).
Google and Apple cannot say that if you want to pay, you must use only our devices.
Now, try to use Pix outside of Brazil
Now, try to buy ice cream from street vendors using any crypto coins.
I was several times required to build simple web apps, and it wasn't hard. I used Python, Flask, Jinja2, Bootstrap, and maybe one or two functions from jQuery for AJAX. It was very easy, and I know only Python and basic HTML and CSS. I don't know any JS/TS because I'm a Computer Vision Engineer.
These web apps show tables with data, images with figures, and some controls to request calculations or data from a server. jQuery is used to update the drop-down list according to previous choices.
Some time ago I did research for myself on the topic of GUI libraries/frameworks. Below are some things from this study. The modern tendency for all frameworks to use 2D rendering engines to draw all widgets from scratch instead of using platforms' widgets and APIs. Usually, they use Skia.
I can recommend reading blog posts by Raph Levien [1], who is one of the founders of Linebender [2] and Xilem [3]. Xilem is a Rust cross-platform GUI framework intended to use GPU for rendering in the first place. Earlier, they tried to develop their own GPU meta-API, then they switched to WGPU [4] and developed their own 2D rendering engine instead of using "standard" Skia, like many others.
JetBrains develops Compose Multiplatform [5]: using the same Android API - Jetpack Compose, but for many platforms and in Kotlin. They also use Skia but are considering switching to Impeller (a new 2D rendering engine in Flutter) because of performance issues in Skia with shaders compiling.
Slint [6], was founded by 3 guys who's been working for a long time in Qt Group. It is written in Rust but uses DSL to develop GUI. As I understand, they try to solve the main fundamental issues in Qt.
MAUI, as I understand it, was Xamarin.Forms and now Microsoft tries to develop it.
Another interesting thing. There is a part of each GUI framework, a system to automatically place widgets on the screen - layouting, something like a grid, table, and so on. Several years ago, Apple had a big boom in the implementation and use of the Cassowary algorithm [7] in their GUI frameworks. It was a promising, strong math-based academia-proof algorithm, but in real life, as I understood, it can not work reliably in many edge cases [8], so Apple quietly removed it and implemented Flexbox [9].
More approaches: to use game engines as GUI frameworks: Unity, Unreal, Godot, etc.
Fun fact: Blender has its own GUI library, which is implemented on top of OpenGL.
PS If you find anything interesting, I would appreciate it if you share.
PPS As a way to find more, I can recommend using search in HN, Reddit, and GitHub. Try to find projects that people develop to solve something and look at what ideas they use.
[1] https://raphlinus.github.io/
[3] https://github.com/linebender/xilem
[4] WebGPU implementation in Rust, used in Firefox and many other projects, can be used as a separate library, cross-platform, meta-API over Vulkan, DirectX, Metal etc
[5] https://www.jetbrains.com/compose-multiplatform/
[6] https://github.com/slint-ui/slint
[7] https://en.wikipedia.org/wiki/Cassowary_(software)
Sometimes, I miss paper-based magazines, and when I have an opportunity to stay and look through newsstands in supermarkets or bookstores, I do, and sometimes buy. Harvard Business Review is good one. Wired is sucks, one of the worst.
Does anyone know good magazines about tech/programming/engineering?
I found CODE Magazine [*], which looks promising, but it is primarily about C#/.NET.
Once upon a time there was Google Web Toolkit (GWT), ASP.NET WebForms. What approach do they use, the same as FastUI? Now there is Blazor for C#, how does its approach differ from FastUI? I think they all promise to write UI for Web without writing JavaScript code.
On the crowdfunding page [1] they revealed causes why they decided to run the campaign:
Our goals with crowdfunding the development of OpenCV 5 are several: 1) To make OpenCV 5 our biggest release ever, with the most community participation of any version so far, 2) To prove to other struggling open source projects that crowdfunding is a viable option over seeking corporate donations exclusively, 3) To create a sustainable method of fundraising that OpenCV can return to year after year, making the overall organization and project more robust to global instability.
Recently, due to global conflicts and instability, OpenCV has lost several core team members, and this has slowed progress on both new feature development and the handling of bugs and requests in the GitHub repository.
Crowdfunding development of a big Open Source project like OpenCV is a big move. We aren’t aware of ANYONE doing it at this scale. As a non-profit organization, we are constantly fighting to find funding, but we always find support in our community. Instead of spending our time chasing big checks from billionaires, we are putting that community front and center.
[1] https://www.indiegogo.com/projects/opencv-5-support-non-prof...
Yes, it is totally relevant. Creating a product that processes images/videos/photos typically requires composing some small models, applying classical functions to work with polygons, edges, transformation matrices to rotate/shift coordinates and pixels, resizing, working with colors, working with bright/contrast curves, open/write files, classic filters, etc. PyTorch and Tensorflow don't have this or use in their examples OpenCV functions to do it.
Yes, it is really important piece of software. I use them time to time. I donated to them.
Location: Brazil
Remote: Yes (4+ years)
Willing to relocate: Yes, I am easy going
Technologies: Python and Golang, Numpy, Pandas, SciPy, Scikit-learn, PyTorch, OpenCV, Pillow, Matplotlib, Docker, Git, Bash, GDAL
Résumé/CV: https://www.linkedin.com/in/ilirium/
Email: [kalimulin.jbs] located in the [gmail.com] service
Seeking: full-time or contract, I have my own company
Key results:
– Create, train and tune DL models
– Statistical processing and data analytics of times series from IoT sensors and satellite images
– Remote sensing and computer vision: georeferencing, warping, image filtering and processing, object recognition, interpolation
– Benchmarking and speed up: async, threads, multiprocessing, multidimensional array algorithms with Numpy
– Developed backends for data analyzing services and prediction API in Python and Golang
– Dashboard for comparing data sources by different statistic metrics
– Deployed services to production and staging with Docker Compose, GitLab CI/CD and Bash
Fields: AI, Data Science, Machine Learning, Deep Learning, Computer Vision, Backend. Geospational data and IoT sensors (weather and environmental), satellite images, gridded/array data and time-series.
Location: Brazil
Remote: Yes, or Hybrid or Office
Willing to relocate: Yes, it is possible
Technologies: Python and Golang, Numpy, Pandas, SciPy, Scikit-learn, PyTorch, OpenCV, Pillow, Git, Bash, Jupyter, Matplotlib, PyCharm, Docker, GDAL
Résumé/CV: https://www.linkedin.com/in/ilirium/
Email: [kalimulin.jbs] located in the [gmail.com] service
Fields: Data Science, Machine Learning, Deep Learning, Computer Vision. Geospational data and IoT sensors (weather and environmental), satellite images, gridded/array data and time-series.
Key results related to hard skills:
– Create, train and tune DL models
– Statistical processing and data analytics of times series from IoT sensors and satellite images
– Remote sensing and computer vision: georeferencing, warping, image filtering and processing, object recognition, interpolation
– Benchmarking and speed up: async, threads, multiprocessing, multidimensional array algorithms with Numpy
– Developed backends for data analyzing services and prediction API in Python and Golang
– Dashboard for comparing data sources by different statistic metrics
– Deployed services to production and staging with Docker Compose, GitLab CI/CD and Bash
Location: Brazil
Remote: Yes, or Hybrid or Office
Willing to relocate: Yes, it is possible
Technologies: Data Science, Machine Learning, Deep Learning, Computer Vision. Geospational data and IoT sensors (weather and environmental), satellite images, gridded/array data and time-series.
Résumé/CV: https://www.linkedin.com/in/ilirium/
Email: [kalimulin.jbs] located in the [gmail.com] service
Main tech stack: Python and Golang, Numpy, Pandas, SciPy, Scikit-learn, PyTorch, OpenCV, Pillow, Git, Bash, Jupyter, Matplotlib, PyCharm, Docker, GDAL
Key results related to hard skills:
– Create, train and tune DL models
– Statistical processing and data analytics of times series from IoT sensors and satellite images
– Remote sensing and computer vision: georeferencing, warping, image filtering and processing, object recognition, interpolation
– Benchmarking and speed up: async, threads, multiprocessing, multidimensional array algorithms with Numpy
– Developed backends for data analyzing services and prediction API in Python and Golang
– Dashboard for comparing data sources by different statistic metrics
– Deployed services to production and staging with Docker Compose, GitLab CI/CD and Bash
Location: Brazil (South America)
Remote: Yes, or Hybrid or Office
Willing to relocate: Yes, it is possible
Technologies: Data Science, Machine Learning, Deep Learning, Computer Vision. Geospational data and IoT sensors (weather and environmental), satellite images, gridded/array data and time-series.
Résumé/CV: https://www.linkedin.com/in/ilirium/ DM or email for formal resume in PDF
Email: [kalimulin.jbs] located in the [gmail.com] service
Main tech stack: Python and Golang, Numpy, Pandas, SciPy, Scikit-learn, PyTorch, OpenCV, Pillow, Git, Bash, Jupyter, Matplotlib, PyCharm, Docker, GDAL, PROJ, QGIS, GeoTIFF, HDF5, GRIB, NetCDF4, WKT, optical flow.
Weather and environment data: NOAA, GFS, ERA5, WRF, solar radiation, weather radars, SILAM, WaveWatch and etc.
Key results related to hard skills:
– Create, train and tune DL models (classification, segmentation, prediction tasks)
– Statistical processing and data analytics of times series from IoT sensors and satellite images
– Remote sensing and computer vision: georeferencing, warping, image filtering and processing, object recognition, interpolation
– Benchmarking and speed up: async, threads, multiprocessing, multidimensional array algorithms with Numpy
– Developed backends for data analyzing services and prediction API in Python and Golang
– Dashboard for comparing data sources by different statistic metrics
– Deployed services to production and staging with Docker Compose, GitLab CI/CD and Bash
Key results related to soft skills:
– Wide experience of remote work, also before the lockdown
– Defended a Ph.D. degree on time
– Presented research results on scientific conferences (more than 20)
– Retrained myself from a research fellow in the electronics industry to the data scientist in IT
Hi Tim, thanks for the podcast and the blog post.
1) Could you share examples of bad, good, and open-source Japanese software? Outside of the gaming industry and except for Ruby.
2) Does they have some kind of the hacker culture?
3) May you know links which answer question: Why Europe/Brazil/Turkey/Russia/etc failed to build well-known and international software companies, FAANG-like.
Location: Brazil
Remote: Yes, or Hybrid or Office
Willing to relocate: Yes
Technologies: Data Science, Machine Learning (ML), Deep Learning (DL), Computer Vision (CV). Geospational data and IoT sensors (weather and environmental), satellite images, gridded/array data and time-series
Résumé/CV: https://www.linkedin.com/in/ilirium/
Email: [kalimulin.jbs] located in the [gmail.com] service
Key results related to hard skills:
– Create, train and tune CNN/RNN models (classification and segmentation tasks)
– Statistical processing and data analytics of times series from IoT sensors and satellite images
– Remote sensing and computer vision: georeferencing, warping, image filtering and processing, object recognition, interpolation
– Benchmarking and speed up: async, threads, multiprocessing, multidimensional array algorithms with Numpy
– Developed backends for data analyzing services and prediction API in Python and Golang
– Dashboard for comparing data sources by different statistic metrics
– Deployed services to production and staging with Docker Compose, GitLab CI/CD and Bash
Key results related to soft skills:
– Wide experience of remote work, also before the lockdown
– Defended a Ph.D. degree on time
– Presented research results on scientific conferences (more than 20)
– Retrained myself from a research fellow in the CAD/EDA industry to the data scientist in IT
Main tech stack: Python and Golang, Numpy, Pandas, SciPy, Scikit-learn, PyTorch, OpenCV, Pillow, Git, Bash, Jupyter, PyCharm, Docker, GDAL, PROJ, QGIS, GeoTIFF, HDF5.
Location: Brazil
Remote: Yes, or Hybrid or Office
Willing to relocate: Yes, prefer USA or Canada
Technologies: Data Science, Machine Learning, Computer Vision (CV). Geospational data and IoT sensors (weather and environmental), satellite images, gridded data and time-series. Python and Golang, Numpy, Pandas, SciPy, Scikit-learn, PyTorch, OpenCV, Pillow, Jupyter, PyCharm, Git, Bash, Docker, GDAL, PROJ, QGIS.
Résumé/CV: https://www.linkedin.com/in/ilirium/
Email: [kalimulin.jbs] located in the [gmail.com] service
* Key results:
– Improved RPS x7 for an in-house big data storage (terabytes)
– Create, train and tune CNN models (classification and segmentation tasks)
– Statistical processing and data analytics of times series from IoT sensors and satellite images
– Developed a machine learning services for data analyzing and prediction with API
– Wrote data processing pipelines
– Intensive usage of multidimensional features to speed up calculation in Numpy
– Wide experience of remote work, __ before the lockdown __
* Strong abilities for self-education: I retrained myself from a research fellow in the electronic design automation industry to the data scientist in IT.
Digital nomad (it helps me to develop an open multicultural mind and to consider aspects of technology and life from different points): currently Florianopolis (Brazil), spend 1 season in India, 2 seasons in Europe, 3 seasons in Turkey.
Try to update your Python environment and also install ALL recommend dependencies for Pandas (and for Geopandas if you use it). I had one old env in Anaconda (Python 3.9.12, Pandas 1.4.2) and then created a new one in Mambaforge (Python 3.10.6 with Pandas 1.5.0). It gives me speed up one of experiment project from ≈30 min to ≈5 min. Use Python code only for glue and leave extensive calculations for C/C++ code. Pandas and Geopandas have the recommended C/C++ dependencies which dramatically speed up calculations and in new versions as I guess they improve integrations with these dependencies.
PS I also recommend to every one to use Mambaforge instead of Anaconda, because it uses Mamba dependency solver written in C++ and it is in orders of magnitude faster.
One more reason why: there is no job for Juniors, Middle and Senior. Only such as 10+ years of experience with a narow specialization, this is not for Senior, it is for Senior++ ;)
2 decades ago, there were plenty of jobs for newcomers: shareware desktop apps for Windows, growing CAD/EDA/simulation tools, small and middle-sized games, and enterprise apps. Today there is no room for desktop apps for Windows, CAD/EDA/simulation seems stagnant, games require Unreal/Unity/Godot with their own languages, and enterprises used C# and Java.
Embedded C++ positions differ from "General C++ Senior++ Engineer". Yes, they are welcome for novices, but they use their own subset of C++ of an outdated language version, and you cannot grow with them, because the size of a project is small and after 10+ years you cannot be Senior++ with required rare specialization (and of course low salaries).
What else leaves for novices to get more experience? Only writing and supporting your own open-source project with a few users (or no user base at all) during a decade… So, it is not an option for many people.
Location: UTC-3, USA, Canada, California, Toronto, Ottawa
Remote: Yes, or Hybrid/Office
Willing to relocate: Yes, prefer USA, Canada and maybe Japan or Korea
Technologies: Data Science, Machine Learning, Computer Vision (CV). Geospational data and IoT sensors (weather and environmental), satellite images, gridded data and time-series. Python, Numpy, Pandas, SciPy, Scikit-learn, PyTorch, OpenCV, Pillow, Jupyter, PyCharm, Git, Bash, Docker, GDAL, PROJ, QGIS.
Résumé/CV: https://www.linkedin.com/in/ilirium/
Email: [kalimulin.jbs] located in the [gmail.com] service
* Key results:
– Intensive usage of multidimensional features to speed up calculation in Numpy
– Developed a machine learning services for data analyzing and prediction with API in Python, Pandas, PyTorch, Numpy
– Create, train and tune CNN models (classification and segmentation tasks)
– Statistical processing and data analytics of times series from IoT sensors and satellite images
– Wide experience of remote work, __ before the lockdown __
– Defended a PhD degree on time (only 14% of graduate students go on time to defend in the country by official statistic)
– Presented research results on scientific conferences (more then 20)
* I retrained myself from a research fellow in the electronic design automation industry to the data scientist in IT.
Digital nomad (it helps me to develop an open multicultural mind and to consider aspects of technology and life from different points): currently Florianopolis (Brazil), spend 1 season in India, 2 seasons in Europe, 3 seasons in Turkey, from Russia.
For my cases, when during R&D (data science for weather/geo/agro) I rarely need to work with formulas in symbolic representation, Sympy works enough. Cases like combining 2 equations into 1, simplifying, defining numeric coefficients, and printing formulas in TeX representation.
I don't speak Portuguese and I use a translator to read it. So maybe it is wrong, but as I found in the link which you mentioned, the tax-free limit is 1000 USD (arrival at the country by air or sea) and 500 USD (arrival in the country by other means of international transport).
My main reason to surprising is the size of the limit. For example, if a person buys a laptop that cost 2000 US, he/she must pay additional fees. Or another situation, when foreign visitors bring their stuff (e.g. MacBook Pro or photo equipment) they also must pay fees. It is weird and awful.
I am from Russia. I am not an expert in import regulations, but as I see tips for travelers, the limit is 10000 USD (50 KG) by air for personal belongings and 500 USD (it is low and it was a surprise for me) by other transports.
In general, the customs fee size in Brazil for hi-tech devices for me looks pretty outdated, because I can see that there are only a few local brands and I don't know but maybe they just use OEM devices from China. And, of course, as I understand, not anyone international computer supplier who does localize their manufacturing in Brazil.
And, as I know, many EU residents when they visit the US buying laptops and phones, because it is cheaper and more models are available :)
Thanks. Is it only for Brazilian citizens? When I came to Brazil with my equipment and belongings, no one asked me to pay fees.
As I know, the tax-free allowance is about $50 and it works only when a parcel is handled by Correios (the national post), for other logistical companies (like FedEx) you will pay taxes and customs fees (practically once it cost me +100% to the raw price from Amazon US for a parcel below $50).
[I thought a little later] Maybe you mean for orders from international e-commerce stores? Because paying for taxes and customs fees for goods which you bring by your hands in your luggage during traveling by flight sounds really strange. Of course, I mean the realistic amount, not gray import like bringing dozens of iPhones in bags... For example, as I know, in Russia it was about the equivalent of $10000 in cash.
Do you have to pay import/customs fee only when you bring good from Paraguay? What is about other countries? Is there a special regulation or a law?
Bom dia, tude bem, andrecarini! :)
Currently I also live in Brazil but I came from another country.
I also realized that in Brazil custom fee + taxes for abroad equipments are too high, a custom fee is 60% (yes, sixty percents!), the reader can look prices for Apple devices in their site for Brazil. When I compare prices between US and Brazil Apple sites, I found that difference is about 100%!
But, you known, there is some options. I asked my Brazilian friends and they said that you can go to Paraguay (especially a city near waterfalls) and buy goods there. It is seems Paraguay don't have a custom fee only taxes. You can use https://www.comprasparaguai.com.br/ to compare prices. Brazilian citizens do not need a visa.
Also, I found that sellers on local Amazon and Mercado Livre also sell goods which are much cheaper, then in officials stores. For some goods prices are about like in US Amazon.
Other options are to travel to other countries, like US, Costa Rica, Ecuador or ask you friends or family members from abroad to buy and to bring to you.