mystique

We at mappes.io are doing something similar in industrial domain. Our knowledge graph has few layers

Products (Raw Material <> Application use) Products to company (Supplier <> Buyer) People connected via companies and products

We are already seeing benefits of this in being able to easily discover new connections across products and companies. Our focus is right now on few verticals in manufacturing sector and hope to expand to wider manufacturing space at some point.

Sure that may be the case but there is still some argument applied to both decisions. What's the main difference between one vs. the other that makes decision to get a medical procedure not legal but decision to not wear a mask legal?

Can someone explain in non-legal way why the court believes an individual has right to decide whether they can wear a mask or not but not whether they can have a medical procedure or not? What's the main difference between these two?

Are these conclusions correct though? They show the electoral map and the districts that were lost. Those are more rural and more traditional industry focused areas.

It is more likely that coastal Democrat's income and education was always higher than rural leaning counties and remained at par between these two time periods. To state that this materially changes the Democratic party is not right.

Full disclosure : I work at the company mentioned below

There are products out there like this. There are emerging tools like Syntasa (https://www.syntasa.com/) that have all of these integrations baked in to allow you to grab exact same data from various SaaS tools you are using (like Analytics, Marketing automation etc) and have templates for various analytics activities like attribution or LTV or propensity modeling and integration points back to activation points in your Marketing stack. This allows you to perform many advanced analytics activities, customize it for your business without needing to hire a very large team. In fact, even large teams find it more useful to use a tool like that so they can spend their time worrying about their objectives instead of spending months gluing everything together.

Like OP mentioned, CDPs and Marketing automation are glue tools in a way but they have superficial data and are not meant for in-depth analysis.

This article is suggesting that higher than normal number of people who are on Autism spectrum explains the gender imbalance in tech industry. Further it is somehow attributing this to Damore's reasoning about his memo. There are a lot more normal people in the industry though - surely all the woes directed towards tech industry are not due to folks on the spectrum?

Having worked with normal devs and "brogrammers", I have a hard time accepting that the "brogrammer" that is constantly belittling others on the team, second guessing other's work output is on the Autism spectrum. Wouldn't someone on the Autism spectrum not want to draw everyone's attention towards themselves and just do their own work?

I had a friend in grade school who was on the Autism spectrum. From what I remember, he was awkward, self-aware, genius, not looking for attention - but he was never a jerk and if he knew he was making someone uncomfortable he would stop. To suggest that people on Autism spectrum are misunderstood as brogrammers is a stretch.

This is my story basically for many years. I've found having side project is even tougher as a mom than a dad (SO has been better at carving time for himself than me).

What I've found works is to be consistently working on same side project over time, dedicate some me-time and set small goals. Instead of trying too many new things, stick to same side project for months at a time. Also "book" a few hours every 2-3 weeks for myself and go work at a Starbucks instead of staying home. Finally set small achievable goals; what I would want to do in 1 day, spread that over a month.

It's easy to dismiss the push for diversity by saying that women themselves do not want to go into tech. I used to believe the same when I saw most of my girlfriends opt for non-tech majors - their reason being this is not my cup of tea. Made me feel superior for not only choosing tech, but also being successful academically and later in career.

Now that I have kids, I realize how early the push for conformation to some societal ideals starts. 3-4 yr old kids in day care already have this notion of how girls are not supposed to play with boys, they should look pretty all the time, they should not play rough with each other or other boys. Girls in 1st grade paying more attention to how their hair looks than how good they are reading/writing. I remember that many of my girlfriend's families passively discouraged them from going into male dominated majors, saying it would be easier for them in the long run.

Once, in a line at gamestop, I saw an altercation between a pre-teen girl and a pre-teen boy. The pre-teen boy was adamant that the girl should not be allowed to purchase a game he thought was meant just for boys. Neither of them knew each other. The girl's father interjected and made it clear to the boy that she can choose to play whatever game she wanted and it was not his job to keep her out of it. This is not abnormal in gaming world, grown men even today scorn on female gamers.

IMO, the best way to combat this problem is to have parents actively push their daughters into tech education from early years. Dads spending time with their daughters and giving them confidence that they can be as good as boys will work wonders. And I say Dad specifically because when the most important man in a girl's life shows her that she is as capable and can compete against other boys who will eventually become men, she cares less about fitting societal norms.

Yes. I have used its material and found it helpful.

Machine learning: recommender systems and dimensionality reduction also looks good but it starts in July. If you are already familiar with dimensionality reduction techniques the first one should be enough to get you going. There are other self paced courses on dimensionality reduction on coursera too.

There is a good course on coursera just for recommendation systems. Evaluation of different models is something many algorithm oriented posts don't talk about but is covered well in this course.

How is this different than http://www.foiaxpress.com/ ?

A decent CS undergrad degree decade ago included abstract math concepts. I took Engineering math, Information Theory, Numerical analysis, Probability, Simulation in my sophomore and Junior years. NLP and AI were electives in Senior year.

As a Junior, we were building toy programs that do Operations research type of work - solving linear equations via various matrix operations, design optimal queue processes based on poisson process.

Assuming a software engineer is a CS undergrad, he/she most likely has good footing to learn more by themselves.

Simple use cases that were more complicated

- We wanted to collect files from various locations and push that in hdfs. Nifi seemed like a good way to build a self service setup. Once we setup source and sink, if we read + processed + removed file from destination, Nifi copied it again. We did not have control of always removing source files as soon as it was copied on destination. Components we used were GetFile, PutFile and some options in Conflict Resolution settings for these components

- Inspect file name, run a script to generate new subdirectories for partitions and place the file in appropriate partitions. Attaching a script was easy. Changing the destination path on the fly was not.

Some Complex cases - there are other ways to do this but setting this up in Nifi was a breeze

- Set up file collection from ftp, sftp, file copy from 20+ locations. This was painless, few minutes per source

- Add REST interaction within data flow easily

- Read CSV files and convert to Avro/Sequence files

- Read files and route part of data to different processors

We also ran into some strange bugs where Nifi got stuck in some type of loop and kept copying data over and over again.

We were able to do all this testing in 2 weeks. Give it a shot, it might work out for your use case.

We were wowed by NiFi when we looked at it originally. Once we put it in Local env to build test flows, we found that the most complex tasks for data flows were fairly simple to setup. And the simplest tasks ended up requiring complex workarounds because the system was trying to be extra smart about what it was doing. In the end, we decided not to use it in Production due to the 80/20 split of simple/complex tasks we had.

Hopefully it's better than what it was in Jan/Feb timeframe.

Have you looked at Rundeck? http://rundeck.org/

It has rest based interface to do everything + UI to make it easier when you need to do one off things. It allows you to control execution across multiple nodes. Good thing is you can setup your processes by tag of a node in Dev and export the project as is on Production and it will still work.

It's a runbook automation tool but can be used like in place of chronos, oozie, spark job server etc.

I attributed a lot of this 'effectiveness' to developers who are lazy.

There have been times when I automated things and that was because I wanted to avoid repetitive work. I have never been able to automate something I had not manually done 2-3+ times, but once I did I knew what I needed to do to avoid manual work and make the process error free for next time same operation was needed. If someone else was dictating what they needed, I would not have been able to build my tools as effectively.

To me, a good developer is a "maker", who builds tools. For us the raw material is a computer and some free time, which makes experimentation and evolution of our tools easier compared to other professions.

Ancient Hindu monks have pondered on the question of consciousness forever. Ancient Hindu texts elaborate on differences between Self, Consciousness, Experiences, Knowledge. Some of this is intertwined with the Hindu concept of souls. All of Hindu philosophy is based on the concept that Self (pure consciousness) and Brahman (total reality, universe) is the same if we dig deeper and our goal as a human being is to find that union. In Sanskrit, the term itself is similar to "same yet different" - Advaita Vedanta (http://en.wikipedia.org/wiki/Advaita_Vedanta)

Lot of spirituality and even if you do not believe in those religious teachings, monks who wrote those texts thousands of years ago were conscious and questioning their consciousness.

MapReduce and RDBMS are apples and oranges - both are good at what they do and are effective within their own use cases. One allows you to handle any type of data and manage it whichever way, another allows you to understand your data if you can live within some defined structure. It is silly to suggest to use MapReduce to power a dashboard with sub second response time. Same way, it is silly to suggest using MPP or RDBMS like techniques for processing highly unstructured or even semi structured content.

Apache Spark is getting close to being able to do both, but still as a developer building a data stack, I would not inspect terabytes of data every single time if 80% of questions can be answered by looking at data once and saving summarized results in relational format.

I thought Hadoop vs RDBMS was a fight settled may be 4-5 years ago! Amusing to see it being raised at this time.

Good list. I am new to Machine Learning with only ~1 year of real work and sometimes I slip and make one of these mistakes.

I have a question on #7. I have not used the co-efficients to mean feature importance but some times get tempted to use them. How do you explain which factors are the most important factors behind some outcome to non-stat people?

We had to do something similar in our real world example where we had the labels but were unsure if the labels were truly accurate or not.

We used a technique similar to LSA.

Our first step was to build a bag of words and construct a scaled TF off of that. Then we verified the label for about 10% of the data and used that as our training set. Using cosine similarity (which we calculated using matrix multiplication of tfs) we found top n labeled documents that were similar to document in question to decide the label of remaining 90% documents.

Once we had this dataset we ran it against logistic regression as well by training on same 10% and use remaining to find the label. Interestingly document similarity was only slightly better than logistic regression. Logistic was 10 times faster.

I think this approach worked for us because we had somewhat of a mutually exclusive set of words for one or other label. This may not work in sentiment analysis where same word can have different meanings depending on surrounding words. N-Grams and then TF on it might help in that case.

I have been thinking about this for a while now too.

I feel more troubled by the fact that folks in non-AI field don't see this as an imminent problem. They don't realize that with current advances in computing power that enables us to store more and more data cheaply, how much can be learnt from it and how much can be automated.

There don't seem to be any easy solutions. Training/teaching folks new skills would help but will not solve this for every person affected. I suspect the collateral damage from this will be big.

In the lifetime of a woman, how many maternity leaves does she take? On average it would be twice and very few women I know have taken more than 8 weeks. The way you put makes it sound like women take 6 months+ time off and everyone else has to pay for it.

I agree with squozzer, you should consider expanding your activities beyond folks who know you personally.

I really like my work and I work a lot. I have worked in early-mid stage startup and now am working with a pre-series A stage startup. I have never shied away from asking for more responsibilities or climbing up the management chain. I am generally known as the more assertive person in the organization. But when it comes to compensation, I cannot bring it up. Not sure if it has to do with being a female.

It would be ideal if there was a 'go to' person who can push me more in negotiating for better compensation.

"All men are created equal" except if they happen to be non-americans

As a female and ambitious person whose first pregnancy wasn't at a "convenient" time, I totally get it.

I wasn't the CEO of the company I worked at, but Architect (and Director) and we were working on major overhaul of our product. My son was born right around the Release timeframe. All I could do was design and plan and prepare my team for any last minute emergencies. Like her, my energy levels in last 2 months also were not what I wanted. Everyone in my team had worked our asses off and I felt guilty of not being productive enough in last 2 months and being away right when they would need me if things went downhill. Many of the design decisions were mine and if things went wrong I did not want my team to be hounded by our customers and execs.

Being an employee who is responsible only for himself/herself vs. being responsible for a team(s) is different and if you were not able to time your absence in the right way, it feels even worse.

I have come to realize that our cultural norms have a big role to play in how we (females) treat ourselves. We don't expect/want any special favors at work, but we also expect ourselves to do everything our moms and grandmoms did. It's all about being able to figure out what makes us happy and work to achieve that. It took me a while but I know that I will be miserable if I did not work the way I do and that will not be good for my son either.

For me, it's dejavu again as I just learned today that I am 3 months pregnant. And I joined a startup 4 months ago, and we drafted up a release plan of our first major version to be 6 months from now. Go figure.