Explainer

Data modeling: the hard part is knowing what to build

Building has become easy. The hard part is knowing what to build. Here is how a shared data model becomes the foundation for the whole business.

By Steffen Sande8 min read

Building a table on a modern data platform no longer takes long. What takes time is knowing what it should contain. Here is why data modeling is more about understanding than technology, and why a shared model for the whole business pays off.

Ten years ago, a new dataset could take weeks to put in place. Someone had to set up servers, configure loading tools and make sure the jobs ran in the right order.

Today the platform scales itself, pipelines work out the order on their own, and an AI assistant can write the first version of a transformation in minutes. Yet many teams still find that delivery is slow.

The bottleneck has moved. It is no longer in the building, but in the question that comes before it: what exactly are we supposed to build?

Building is no longer the hard part

Give an experienced data engineer a clear description of a table, and it is often built the same day. The columns are known, the keys are decided, and what each column means is written down. What remains is fetching the data and writing the transformation, and that is the part the tools have become good at.

Without such a description, the engineer has to guess, ask around, or build something that seems reasonable and hope it hits the mark. The table gets built, but often several times, because only afterwards does it become clear what the business actually needed. The fastest route to a finished table therefore rarely runs through the code. It runs through agreeing on what the table should be.

The hard part is knowing what you need

Imagine an online shop building a data platform. The starting point is concepts everyone in the company knows: customer, order and product. Even here, there are questions that must be answered before anyone builds anything:

  • Is a customer a person, a household or a company? And what happens to the orders when two customer accounts are merged?
  • Can an order contain several products? If so, does the price belong to the product or to the order line?
  • When does an order count as sold? When it is placed, when it is paid, or when it is shipped?

These are not technical questions. The answers lie with the people who know the business, and they often disagree until someone asks the question out loud.

If nobody decides, the code decides for them. The developer writing the load job picks one interpretation of “sold”. The analyst building the revenue report may pick another. Both have done something sensible, but the numbers don’t match, and the meeting that was meant to be about sales ends up being about which number is right.

Once you agree on the answers and have checked them with the business, the building itself is the easy part.

One area at a time, with the whole business in mind

Nobody models the entire business at once. It takes too long, and the business has to wait too long before it gets anything. In practice you build one area at a time: first sales, then purchasing, then finance.

The point is to do it with the whole business in mind. When sales needs a customer table, it is not built just for sales. It is built so that finance, customer service and logistics can use the same table when their turn comes.

This is not a new idea. Kimball Group, behind one of the most widely used methods for dimensional modeling, has for decades recommended combining an overall perspective on the whole business with deliveries that take one business process at a time. What ties them together is shared concepts, such as customer, product and date, which are defined once and reused by every process.

The difference from building for one need at a time is small at first, but large over time. Each new area can build on what already exists instead of starting from scratch.

The silver layer: one shared model for the whole business

Many data platforms are built on the medallion architecture, a pattern Databricks made well known and that is also used in Microsoft Fabric. The data passes through three layers, and each layer has its own job.

  • Bronze holds the raw data as it arrives from the source systems.
  • Silver cleans, merges and conforms the data from the different sources. According to Databricks, the silver layer should give an enterprise view of the business’s key entities, concepts and transactions.
  • Gold holds tables shaped for specific needs and analyses. Because the gold layer models a single business area, many organisations have several gold layers, for example one for finance and one for HR.

This means the shared model naturally belongs in the silver layer. That is where customer, supplier, product and sales order are described once, regardless of which system they came from. The gold layers for sales, purchasing and finance build on the same silver tables, and so get the same definitions.

For the online shop, this means the customer from the web shop and the customer from the customer service system become one customer in the silver layer. When purchasing later needs suppliers and purchase orders, they are linked to the same product that sales already uses.

Why it pays off

A shared model brings three concrete benefits:

  • Faster delivery. Kimball Group points out that reusing shared concepts shortens the time from need to delivery, because you avoid designing and developing the same thing several times. If the online shop wants to analyse delivery performance, customer, product and order already exist. Only what is new has to be built.
  • The same numbers everywhere. When sales and finance take the customer from the same table, with the same definition, their reports show the same numbers. The discussion can be about what the numbers mean, not about which number is right.
  • Data can be combined. Shared concepts make it possible to look across processes, for example to compare what has been sold with what has been bought of the same product.

The gain is small in the first month and grows with every area. The first area takes about as long as before, because the concepts have to be clarified thoroughly. The second goes faster, and the third faster still.

The alternative is familiar to most. Without shared definitions, the platform gradually drifts into a collection of isolated solutions, where each area has its own version of the customer. After a few years there are several customer tables, nobody knows which one applies, and every new need starts with finding out what already exists.

AI needs the business context

Many organisations want AI to answer questions about the business: how did sales go last week, which customers are buying less than before. A language model can write the query, but it cannot know what “sold” means in this particular company.

Research points the same way. Earlier studies have shown that accuracy drops sharply when language models answer questions against real enterprise data, because the table structure rarely tells them which business rules apply. A 2026 study tested three language models on the same retail dataset. When the models were given a short document with the business definitions, accuracy rose by 17 to 23 percentage points for all three.

What AI needs, then, is the same as what people need: definitions of the concepts, knowledge of how the tables relate, and descriptions of what the columns contain. A shared data model with good descriptions provides exactly that.

Where to start

Start with the next area on your list, but look up before you build.

  1. Find the concepts the area touches, for example customer, order and product.
  2. Check whether any of them already exist in the silver layer. If they do, build on them instead of creating new ones.
  3. For the concepts that are new: think about who else in the business will need them, and define them so they work for more than this one area.
  4. Check the definitions with the business, and give every table an owner.
  5. Build what you have agreed on.

Keep the model alongside the code in Git, so it changes in the same process and does not drift from reality. The next area then starts with what already exists, and the foundation grows with every delivery.

Sources