Google Analytics 4 was frustrating, so we built our own analytics service

https://requestmetrics.com/blog/product/google-analytics-4-has-me-so-frustrated-we-built-our-own-analytics-service/

by twapi • 3 years ago

103 98 3 years ago

riku_iki• 3 years ago

there are many other analytics products besides GA?..

Yea the market is rammed full of privacy based analytics tools. I guess the market is pre-validated but it is probably a lot of work to break out in that market.

Kye• 3 years ago

Have you ever noticed the large number on your content blocker indicating how many things were blocked? Most of those are analytics.

konha• 3 years ago

Most of those are ads and targeting.

doctorpangloss• 3 years ago

Trying to visit this URL gives: “Safari can’t open the page. The error was: The URL was blocked by a content blocker.”

This is the real obstacle to analytics.

vntok• 3 years ago

A very typical example of the NIH syndrome.

There really isn't much to learn from this article apart from that feeling about "we did it in-house instead and it does 60% of what the dumb original solution would do. Also we need to maintain it all ourselves now.":

Not invented here (NIH) is the tendency to avoid using or buying products, research, standards, or knowledge from external origins. It is usually adopted by social, corporate, or institutional cultures. Research illustrates a strong bias against ideas from the outside

(https://en.m.wikipedia.org/wiki/Not_invented_here)

crisnoble2• 3 years ago

In this case they are creating a service and selling it, so it's not like they just decided to roll their own analytics because they hate GA4.

justinzollars4• 3 years ago

I recently gave google analytics 4 a shot. I actually had the thought that I had more access to information, and my analytics were better in 1998. Its a disaster.

londons_explore2• 3 years ago

Because google analytics 4 sticks to the gdpr...

your analytics in 1998 did not...

habibur• 3 years ago

1998 analytics used Apache log files as the only source.

Wondering if Apache log files are GDPR complient or not.

alex_lav• 3 years ago

A quick google search suggests GA4 is not GDPR compliant

streptomycin• 3 years ago

Nah a lot of it is weird UI decisions. Like to view what browser versions your users are using, in UA you just go to the browser page and click the browser and it lists the versions and number of users. In GA4... good fucking luck, you need to make some weird custom report where you need to select the right parts of your data and tell it whether they are "metrics" or "dimensions" or "rows" or "columns" or "values" or "filters" or "segment comparisons". It's a nightmare. And it's like that for basically everything other than the premade UIs, which feel like they are oversimplified and targeted more towards apps than websites. Generally it feels like I'm using a product made by people who hate the Internet and wish all their website users would go away, but hey it's free...

gh123man2• 3 years ago

I am also frustrated with Google Analytics but one thing it does very well (that I can't seem to find elsewhere) is counting unique unregistered visitors. If I had to guess this is done with some advanced fingerprinting - but I'd be curious if anyone has found an alternative for these metrics.

Uw7yTcf36gTc• 3 years ago

its a simple 2 year cookie for every browser. it's not that complicated.

whalesalad• 3 years ago

https://xkcd.com/927/ comes to mind

(in all honesty though I would probably do the same thing - GA is not my favorite thing in the world)

etaioinshrdlu• 3 years ago

What analytics tools do people like here?

eatonphil• 3 years ago

Fathom and plausible are nice.

gregors• 3 years ago

I've been pretty happy with plausible analytics. They have a hosted solution as well.

https://github.com/plausible/analytics

sccxy• 3 years ago

self hosted plausible

acaloiar• 3 years ago

Both self-hosted and hosted Plausible are a good choice.

epoch_100• 3 years ago

Shynet

okamiueru4• 3 years ago

Not sure how it compares, I set up self hosted matomo (originally named piwik) and it's been running without issue since 2013 or so. https://github.com/matomo-org/matomo. Only useful if what you're running is on php. Probably would've picked a different stack today.

XCSme2• 3 years ago

Only the server where the platform is installed has to be PHP, the tracked websites can be anything.

What is wrong with PHP? Most hosting companies support it by default or come with LAMP images, so installation is often trivial.

modeless2• 3 years ago

I just enabled Cloudflare web analytics on my site rather than try to figure out Google's new thing. All I care about is tracking visit trends and referrers, and that's actually way easier in Cloudflare's UI so I'm happy.

One nice thing about it is that there are some pages on my site that I never bothered to add Google Analytics to, but Cloudflare adds their tracker automatically to every page. It couldn't be easier, which is great as analytics is not something I want to spend any time on.

urda• 3 years ago

I think I will be checking out Cloudflare too since I'm unhappy with GA and I am not eager to run my own analytics platform.

... but if I wanted to run my own platform I wonder what other people are using.

carlosjobim3• 3 years ago

My experience with cloudflare is that it counts every bot as a visitor, so Facebook pinging your site 5000 times in a day shows as 5000 United States visitors - not so useful to analyze. Is it different nowadays?

openplatypus4• 3 years ago

Wide Angle Analytics

toddgardner4• 3 years ago

Request Metrics, obviously!

https://requestmetrics.com

ROFISH• 3 years ago

Because it stores data on a user's computer, isn't using localStorage to track sessions still considered to be a "cookie" for ePrivacy Directive purposes? To that end, I don't think it's not "no cookie banner" compliant for the law as it exists today.

(Note: It _may_ be compliant for future updates of the directive coming in a couple of years, but iirc that isn't out yet.)

krsdcbl• 3 years ago

just to point it out, laws like GDPR do not mention "cookies" anywhere. It's regulating how you may or may not retrieve and store ANY data that makes an individual identifyable.

The whole "cookie" topic is a dark pattern mostly pushed by ad providers as a friendly word to bypass having to say "may we track everything you do?".

So if it tracks data points that can be used to clearly identify a visitor, or marks the visitor in a way that can be used to personally identify them later, it will need the users consent, regardless of where and how this is stored.

thecopy3• 3 years ago

The parent was talking about ePrivacy Directive which is a regulation for companies registered and operating in EU. That is where the cookie banner requirement comes from.

unmole• 3 years ago

The whole "cookie" topic is a dark pattern mostly pushed by ad providers as a friendly word to bypass having to say "may we track everything you do?".

Go to https://gdpr.eu/ and tell me what their cookie banner says.

openplatypus4• 3 years ago

Correct, access to Terminal Device will likely require consent. Local Storage is such access.

But not every access, or cookie, will require consent.

https://wideangle.co/blog/what-is-consent-under-gdpr

danhak• 3 years ago

Glad I’m not alone in this. It’s rare that an interface makes me feel as dumb and frustrated as GA4

One of the more baffling choices they made was to have “today” and “yesterday” as options on the user reporting tab, even though it can take up to 48 hours to process the data and my dashboard always shows zero users for “today”. Took a while to figure out what was going on there…

Meanwhile the real-time reporting works reasonably well but is limited to a 30 minute window. Some real head-scratching choices were made here.

gh123man2• 3 years ago

Agree on all points. It's extra frustrating because GA exposes some metrics that are hard to get elsewhere like traffic sources, organic search, accurate user metrics, live geo location, etc. The only reason I keep it activated (alongside a more flexible alternative) is because it's free.

The whole UX drives me mad.

chocolatkey• 3 years ago

Exactly. I'm currently building in-house analytics (thanks Clickhouse) due to GA4 as well.

The only way I'm able to use GA4 in any way is with their search box + autocomplete, you can sometimes get a decent query result, then you click the three dots next to the tiny line chart, export as CSV and you get the data.

And for events, I think they want you to pay for BigQuery, so you only get some kind of aggregate count, which is useless. It'a too confusing to figure out if I'm wrong about that.

hrpnk• 3 years ago

In bigger setups, all you want is a data collector and router so that you can feed the data into multiple destinations, depending on the use case. Analytics is just one. Example: https://www.rudderstack.com/ & https://www.rudderstack.com/replace-google-analytics-4-guide...

toddgardner4• 3 years ago

We (request metrics, author) are also using clickhouse. But we go beyond analytics to integrate performance, security, api monitoring, and errors under a single interface. We think of it as “client side observability”.

amelius2• 3 years ago

A great example of a product built without love.

This is what you get when you hire developers to make people click ads.

wg02• 3 years ago

Our industry generally has passionate engineers and detached product managers.

okamiueru4• 3 years ago

"I tried the most used thing for A. Had an issue with it, so we built our own."

Maybe I'm just cynical, but I read this as the outcome of "What's the best way to market this thing we built in an already saturated market with a lot of well made free and open source solutions?"

crisnoble2• 3 years ago

There has never ever been a better time to jump ship from GA. All of your historical GA data will NOT be viewable in your GA4 property. The biggest moat they had was millions of sites with multi year page view data for easy trend comparisons. The moat is now broken and they don't care. GA4 and the forced migration is being handled so bad the only explanation is they would rather nobody use GA anymore.

tommy_axle• 3 years ago

Yep, I just setup Umami (https://umami.is/) yesterday and added it to some properties alongside GA to see how it goes. It's a very simple interface with everything I really need for web analytics so I'm enjoying it so far. I self-hosted so if it sticks around the only thing I might look at is having a replica running for it (already put a high frequency backup in place).

thedanbob• 3 years ago

I hope one day when Google forces us to migrate to Google Next Analytics Product™ my org will finally get fed up and switch to something stable. I’m philosophically opposed to analytics in the first place and would really like to never have to think about it again.

powera• 3 years ago

Is the problem that Google's existing customer base doesn't want the restrictions of GDPR, so Google is tying itself into knots trying to achieve the impossible?

Or is it simply that the concept of "an American company processing data on visitors to a European website" is completely illegal now, and the courts haven't gotten around to spelling it out that bluntly yet?

coderintherye• 3 years ago

People who don't think the forced change to Google Analytics 4 is a big deal may want to consider talking to actual business users of the current Google Analytics (because Google clearly isn't). GA4 is clearly aimed at data scientists. Even as someone who owns data teams it is a frustration to use. Try it for yourself, count the steps necessary to determine many page view counts a certain page is receiving in current "Universal Analytics" vs. the new GA4. That is if you can even figure out how to do it in GA4.

And it's all a shame because Google already acquired a great data science/viz tool, Looker, and that's a perfectly fine place to do data analytics/science rather than forcing it in Analytics.

fourseventy• 3 years ago

GA4 is a dumpster fire. Countdown timer to the forced switch from UA to GA4 -> https://thoughtmetric.io/topics/ga4-countdown-timer

toddgardner4• 3 years ago

Put Request Metrics on your list to credit the post :)

kossTKR• 3 years ago

The GA4 interface along with Ads and some of the Cloud interfaces are so bizarrely complex it's comical.

I wonder what went wrong, but i'm so happy to not trapped in that hellscape anymore.

I remember it could take my team a week to even find the page something was on, or figure out the grotesquely convoluted methodology or naming scheme they had come up with for some simple task from 100 versions of their docs.

tacker2000• 3 years ago

I think the main problem is the way their Material UI concepts are defined and the need for them to use these constraints for all of their products.

They can’t be adaptive and innovative, they need to follow their UI guidelines, so sometimes it ends up being a mess.

Yes, the Google Ads interface is getting worse and worse, i never understand which hierarchical level I am on, its crazy…

openplatypus4• 3 years ago

Not everyone can build or host their solution. And most hosted solutions outright prohibit processing personal data, throwing users and customers under the bus.

Hence, a long time ago we built web analytics, the Wide Angle Analytics. We actually support processing Personal Data when requested.

When building it, we spent more time on compliance than on technology.

leros• 3 years ago

The only thing keeping me from immediately jumping ship from GA is that it's free.

zpeti• 3 years ago

Can someone explain what the rationale is behind this move by google? Theoretically people working at google are fairly smart. But they are forcing a switch to a product that is an order of magnitude worse.

I wouldn’t be surprised if they lose over 50% of GA users with this. Why? Why is it good for google? I don’t get it.

equity• 3 years ago

GA4 has several advantages over UA to account for the evolving privacy landscape. For one, it supports user modeling in ways that are GDPR compliant, and you simply cannot count these users in UA. Apple and the leading browsers have made changes with cookies that have necessitated product changes, but not using fingerprinting as suggested elsewhere.

GA4 also reports on combined app+web data whereas UA did not.

Another distinction is that GA4 has better support for muti-touch attribution, e.g. all of the user ad touchpoints, for ad traffic, whereas UA was primarily last click, or first click if you look at the user acquisition reports.

yosito2• 3 years ago

Just from a UX perspective, the quality of Google products seems to have tanked in recent years. Android, Sheets and Maps seem like they're still best-in-class. Maybe Gmail is still ok too (I don't know, I don't use it). But almost everything else has started to feel completely unusable to me. Even Google Fi is starting to suck.

Analytics is literally the cornerstone of Google's business model. Did Google just get too complacent at the top?

6510• 3 years ago

Maps drives me nuts, the data is truly glorious but I hate the interface

carlosjobim3• 3 years ago

Agreed. Maps is the only product remaining where Google is a leader. There are much better alternatives to GMail and Google Search, but not to maps. I think online maps are only going to become stronger and more important in people's lives, as everything there is real and not machine generated. Apple will never be able to compete until they allow users to add places that are missing.

okdood64• 3 years ago

Google Fi

What Google Fi UI do you use on a regularly basis?

yosito2• 3 years ago

I'm referring to the UX, not the UI. I have Google Fi that I used to use as a backup hotspot while traveling. But then they decided I travel too much and arbitrarily cut off my data one day, so I started buying a global eSIM from Airalo and using it alongside Google Fi for calls/SMS. Then the calls and SMSes started having trouble. I reached out to Fi support, and they didn't seem to comprehend the issue. It took several days to get a response and the response was less intelligent than talking to GPT3.

chrismsimpson• 3 years ago

Check out maxmind.com for a cool IP geocoding database if you’re rolling your own analytics.

crftr• 3 years ago

I currently use the maxmind webservices for client-side resolution in a commercial application. It's reliable but expensive enough that I'm looking for alternatives. I'm currently experimenting CloudFront's location request headers for a massive savings.

Where maxmind is $0.0003 per query. And my CloudFront concept is $0.0000015 per query.

reincoder• 3 years ago

Well, if you want to limit your lookups to country level you can use our free IP to Country database [0]. Feel free to ask what is the catch. It's an easy answer, there is none.

The database is updated daily, there is no quality compromise, it is a database so unlimited lookups and it is licensed under CC-BY-SA 4.0 so it permits commercial use with attribution. It is a subset of our paid city level data. If you want city level data, we have the free tier on the API permits 50k lookups per month.

By the way, I already pinged you on Twitter about the database. :/

[0] https://ipinfo.io/products/free-ip-database

algustionesa• 3 years ago

Google Analytics 4 is not ready for prime time.

I have reviewed many Google Analytics alternatives. You can read about them here: https://algustionesa.com/google-analytics-alternatives/

One thing to note is that many Google Analytics alternatives suffer from data accuracy issues.

They record and count analytics data differently from how Google Analytics does.

Therefore, before fully transitioning to Google Analytics and being surprised by a drop in visitors, it's advisable to use any analytics alternative alongside it first.

toddgardner4• 3 years ago

Author here.

Clearly Request Metrics should make it on your list ;)

noizejoy• 3 years ago

One thing to note is that many Google Analytics alternatives suffer from data accuracy issues. They record and count analytics data differently from how Google Analytics does.

During my years of experience using Google Analytics, I had very limited faith in Google Analytics accuracy. For example, bots routinely poisoned the referrer logs.

So holding them as a benchmark of accuracy seems off the mark for me.

I have the strong suspicion, that nobody has really great bot detection to this day.

Even with Google Analytics, I mostly used the data for rough trend analysis over longer periods, rather than react to every little 5% blip in the data.

For example, it was useful for observing the shift to mobile at differing pace for different types of websites. And that allowed for prioritizing website redesign projects during the 2010s.

And I’m also pretty puzzled at the criticism in your article of Matomo for being too close to how to Google Analytics works, while simultaneously holding Google Analytics as the gold standard to be compared against. That seems rather inconsistent to me.

steviedotboston• 3 years ago

I work with a lot of non-profit Drupal clients and we've switched them from GA to Plausible Analytics because of GA4. It's not free, but it's cheap enough that they're willing to pay in order to not deal with all this bullshit.

Plausible has a really simple interface, and we're able to embed the dashboard within the Drupal admin interface too so they don't even need to log in to a separate service to see their stats.

openplatypus4• 3 years ago

We support selected FOSS and non-profits with a free plan at Wide Angle Analytics, in case you would like to explore alternatives.

digitalengineer• 3 years ago

Change is hard I guess. A 15 + year old product, based on pageviews, that known no bounds in regards to tracking, getting replaced by a new hit-based one that is not perfect, but at least takes GDPR into account. It actually hooks up to BigQuery easily and that can get hooked up to Looker Studio. And it's free. BQ is just a few bucks a month). You can even personalize the main menu. Don't need e-commerce? Just delete it from the menu! Need custom stuff? Build it easily in 'Explore' and place it in the main menu. But, yes you do have to learn the new interface. (edit; typo)

winrid• 3 years ago

Shout out to my little service which tracks time on page: https://watch.ly/

Google Analytics 4 was frustrating, so we built our own analytics service

Related Stories