Many agencies that are using connectors like Supermetrics or Funnel are experiencing performance issues when working with Data Studio once datasets get bigger and the reason is that you are most likely skipping an important step.
What you can do to avoid performance issues is to first pull the data into BigQuery which is Google’s data warehouse and unlike Data Studio it’s actually designed to query and process big datasets so you won’t experience any performance issues.
In a second step you then push out the data to Data Studio, which is designed to visualize data, but ideally you want to be working with finished tables here.
Think of it this way: Google is giving you two highly performant tools that integrate seamlessly. BigQuery to prepare and process datasets and Data Studio to visualize the tables. Two use cases, two tools.
BigQuery also makes it super easy to push your data out to other data visualization platforms like Tableau, Looker, Microsoft Power BI and more if you want to switch things up.
But of course there are other data warehouse tools as well so let’s go over why we recommend BigQuery for digital marketing agencies.
What makes BigQuery the best data warehouse for agencies?
Full disclosure, we do not get paid by Google to recommend BigQuery, our data engineers and developers simply think this is the best tool in the market right now.
Performance: How’s this for speed, you can process 35 billion rows of data in 10 seconds. In other words, plenty! Good-bye performance issues with Data Studio or other data visualization platforms, your broadband speed will be the limiting factor when loading Data Studio reports that draw on tables in BigQuery.
Integrations with your existing stack: Obviously agencies typically already work with the Google suite of tools like Data Studio and Sheets and the integrations are seamless for all major data visualization tools like Tableau, Looker, PowerBI. But there’s more, BigQuery also integrates with leading data pipeline tools like Funnel and even with purpose built two-way data pipelines like Shape ADI that allow you to push optimization changes back to the popular ad networks (here’s a PPC Hero blog post with more on that).
Reliable: BigQuery runs on the Google Cloud Platform and it has an industry leading uptime SLA of 99.99% meaning your data is always safe, replicated and accessible 24/7. Again, if there is an issue your broadband speed is the most likely culprit. As the performance levels above indicate this is an enterprise level product and most agencies will be barely scratching the surface of what BigQuery is built for.
Price: Can be summed up as “bang for buck”. The performance you get out of BigQuery is unparalleled, but it’s surprisingly cheap. We offer a managed data warehouse service for agencies and process terabytes of data daily, yet the BigQuery costs are not a big cost factor. That being said, you are being charged for processing so very inefficient SQL queries that run a lot can cost you so it’s worth setting it up with an expert.
3 key elements for success with BigQuery
1. Technical expertise
Reality check, as with all advanced technologies it’s important not to underestimate the human factor. You’ll need data engineers with a good command of SQL and experience working with APIs and data pipelines to add more data sources and set up automated data flows.
The reality is that larger agencies can afford to have data engineers on payroll and find it easier to attract top talent. That’s why we founded Acuto to give agencies an edge that are on the smaller side but have clients that are no less ambitious and demanding!
To address this problem we offer a managed data warehouse service. We set up your BigQuery data warehouse and manage the data and the data flows for you and help you visualize the data, but you own everything.
2. Minimize maintenance and manual steps
You can connect any number of data sources so they are continuously pulling data using APIs and data pipelines and data integrity is key so you can trust your analysis and automations.
If you want to find out more about the trade-off between building your own API connections or using an out-of-the-box data pipeline you can find out more in this blog post comparing the cost of each option with a real-world scoping example.
Suffice it to say that updating and managing API connections and data pipelines takes a bit of maintenance and that’s included in our managed service so you can focus on working with your clients and trust in your data.
3. Enable your staff to work with the data
Finally, it’s key to prevent a scenario where your agency is overly reliant on a single member of staff so you’ll want to upskill a couple of team members and enable them to use BigQuery in combination with Data Studio.
SQL and Data Studio training for account managers as part of our managed service to upskill key members of your team. It’s not rocket science and a great career development initiative for your staff as well!
How to get started?
You can find out more in our eBook on Data Warehousing for Digital Marketing. Or, if you simply want to have a quick conversation with us about what is required with this approach and what kind of investment you’re looking at, then free to contact us and we’ll set up a quick call.