Category: Managing Change

Start-ups and scale-ups often prioritise quick decisions to maintain their competitive edge, which can lead to shortcuts in data analysis or overreliance on intuition. The impact is often immediate because hasty decisions based on incomplete or improperly analysed data can result in missed opportunities or strategic missteps.

This is particularly true when data is fragmented across silos. Teams simply cannot access or integrate information efficiently. This forces tech leaders to either wait for data consolidation (slowing down the process) or make quick decisions based on incomplete data, sacrificing rigour (accuracy).

This article will address these two primary challenges and offer actionable solutions while solving the other three capital problems in data-driven decision-making. However, this is not our normal day at work. In this case, things just cannot be worse. We are operating in a high-pressure scenario where the company is on the brink of financial ruin and you, as a technology leader, inherited a chaotic environment with poor data processes. The goal is to quickly induce enough order to enable survival, even if perfection is impossible. 

5 Biggest Challenges for Start-up and Scale-up Tech Leaders in Data-Driven Decision-Making

In any given scenario, the challenges are the same:

1. Data Silos and Integration 
2. Data Quality and Accuracy
3. Scalability of Data Infrastructure
4. Talent Shortages and Skill Gaps
5. Balancing Speed with Rigour

But, in our situation, we can’t use the familiar approach and/or deploy common strategies. We need to step up our game. 

1. Data Silos and Integration

Start-ups and scale-ups often adopt multiple tools and platforms quickly, leading to fragmented data spread across various systems (CRM, ERP, marketing tools, etc.). Integrating this data into a cohesive system is complex and resource-intensive. This is especially true if you fail to a) invest in data integration platforms, and/or b) develop a unified data architecture early on. 

In all honesty, a tech leader’s hands are often tied either due to budgetary restraints or late arrival. Consequently, disconnected data sources hinder holistic insights and create inefficiencies in decision-making and you can’t exactly “correct” what’s been done wrong right from the start on short notice.

How to solve this problem?

When traditional mitigation strategies are not viable, you can still take alternative, resource-efficient steps. These approaches focus on leveraging existing resources, prioritising immediate needs and adopting creative low-cost solutions.

1.1. Manual Integration with Pragmatic Prioritisation

Identify the most critical data silos that impact decision-making and prioritise integrating those first. Use lightweight manual processes or scripting (eg, Python, Google Sheets) to consolidate data where automation tools are unavailable.

From that point onward, do the following:

• Conduct a quick audit to map critical data flows and prioritise based on business impact.
• Use basic automation tools like Zapier, Make (formerly Integromat) or built-in export/import features of existing platforms.
• Focus on incremental improvements—address key bottlenecks rather than aiming for perfection.

The outcome of these measures should be partial but impactful data integration for essential use cases without significant resource investments.

1.2. Leverage Existing Tools and Free/Open-Source Options

Maximise the utility of existing platforms and adopt free or open-source tools for basic data integration. Your sequence of actions should be like this:

1. Explore native integrations provided by current software (eg, APIs, built-in connectors).
2. Use free or community editions of ETL tools (e.g., Apache Airflow, Talend Open Studio).
3. Encourage teams to utilise data exports, shared dashboards or reports from existing tools.

This should result in cost-effective integration with tools already in your tech stack.

1.3. Empower “Data Stewards” Within Teams

If you are in a larger organisation, identify key individuals within departments who can take ownership of their team’s data. These people should act as intermediaries to share and consolidate information. 

Now, to make this process as smooth as possible, take the following steps:

1. Designate a “data steward” in each team to document, clean and standardise departmental data.
2. Create simple workflows or templates for data-sharing (eg, shared Excel sheets or cloud folders).
3. Facilitate regular meetings where data stewards align on metrics and share insights.

What you are looking to achieve with this is not only improved communication but also understanding of data across departments without requiring centralised systems. It is a longer walk around, no doubt, but on the bright side, it will create a data processing singularity in the long run.  

1.4. Adopt a “Federated Data Governance” Model

At first glance, this solution seems like it might lead to a pinball effect, with you bouncing from one office to another in a desperate search for that final document. Be that as it may, if you allow teams to maintain control over their own data while introducing light governance structures, it will a) reduce silos, and b) result in shared standards and definitions. However, it won’t happen on its own so to achieve those results, follow this strategy:

• Define a small set of core metrics or KPIs that all teams must report consistently.
• Provide teams with guidelines for data structure, format and reporting (eg, a standard CSV template).
• Finally, use simple collaboration tools (eg, Slack, Notion) for sharing updates and insights.

And there you have it – a fully decentralised yet coordinated approach to data management that minimises silos. Because sometimes, even the government’s bureaucracy turns out efficient. 

1.5. Pilot Low-Cost Data Lake

If — and this is a big if — resources allow for at least minimal investment, pilot a low-cost, pay-as-you-go cloud data lake solution. You want a focused, incremental approach to centralisation without incurring large up-front costs.

This is one of the possible approaches:

• Use tools like Google BigQuery, Snowflake (trial/limited scale) or AWS Athena for specific data sets.
• Gradually migrate the most critical data into the data lake while leaving less critical silos untouched.

Later, during a fast-growth stage, when you get your hands on more resources, this can easily evolve into a full-stack cloud data storage and processing.

1.6. Create a Cross-Functional Data Task Force

As you can assume, this strategy perhaps better fits the onset of the fast-growth stage, but it could also be just what you need in your start-up. This is how it works:

• First, you start by forming a small task force with representatives from key teams to collaborate on solving integration challenges (not a full data team).
• Then, you task the team with regularly consolidating reports or insights and aligning metrics. 
• Finally, they share consolidated data via basic tools (eg, Google Drive, Notion, shared dashboards).

It is an agile team effort that minimises dependencies on expensive tools or specialists.

The core philosophy here is: start small, build incrementally.

In other words, when constrained by budget or timing, focus on solving the highest-impact problems first. Admit to yourself that perfect integration may not be possible, but incremental improvements can still provide meaningful value. By being a bit creative and by maximising existing resources, technology leaders can mitigate the impact of silos without requiring substantial investments.

2. Data Quality and Accuracy

Your most immediate challenge is the all too familiar consequence of rapid growth and that’s a lack of consistent data governance. As you know, this inevitably leads to poor data quality (inaccuracies, duplicates or incomplete data).

The impact can turn out devastating because low-quality data undermines the reliability of insights, leading to poor strategic decisions. Imagine a marketing team missing an entire segment of the target audience or misaligning the core message. Sooner than later, all fingers will point at you.  

On a normal day, you would mitigate by:

• Implementing data validation and cleansing processes.
• Establishing data governance frameworks.
• Regularly auditing and updating data sets to ensure accuracy.

But remember, this is not your normal day. More often than not, technology leaders inherit a chaotic environment with poor processes and must react instead of being proactive. 

Here’s what you can do in such a situation:

2.1. Triage the Data Chaos

Your immediate priority is to identify the most critical areas where poor data quality immediately impacts the company’s survival. Take the following steps:

• Conduct a rapid audit of key data pipelines and processes.
• Focus on revenue-critical systems (eg, billing, sales forecasting, customer data).
• Prioritise data that directly affect regulatory compliance, financial reporting or mission-critical KPIs.

In the end, you will understand where to focus efforts for maximum impact in the shortest time.

2.2. Deliver a Few Quick Wins to Build Credibility

In other words, identify and solve one or two highly visible data issues to demonstrate progress and build trust. Simply fix a problem that has frustrated key stakeholders (eg, cleaning up sales pipeline data or resolving overdue billing errors) and then publicise the success with tangible results (eg, “Resolved 300 duplicate records, improving invoice accuracy by 20%”).

And now you have improved stakeholder confidence and momentum for broader changes.

2.3. Implement a “Minimum Viable Governance”

Quickly enforce lightweight rules to address the most damaging data quality issues without overengineering. This is achieved by:

• Defining non-negotiable standards for critical data fields (eg, customer IDs, transaction amounts, dates).
• Creating simple validation scripts to flag obvious errors (eg, missing fields, incorrect formats).
• Using tools already in place (eg, Excel, SQL, lightweight automation tools like Zapier) for basic cleaning and validation.

If you do everything right, you should end up with an immediate reduction in errors, enabling more reliable decision-making.

2.4. Mobilise a Data “SWAT Team”

This strategy is more appropriate for larger organisations, but it can be scaled down to fit the purpose of a start-up. 

In essence, you assemble a cross-functional, small team with representatives from critical departments to act as a task force. To succeed, this is what you should do:

• Identify power users or, as some call them, “data champions”, from key teams like finance, operations and marketing.
• Assign clear roles: one focuses on cleaning sales data, another on financials, etc.
• Empower them to fix data in real-time and escalate issues to you directly.

The outcome is rapid, team-based problem-solving that restores operational functionality.

2.5. Apply a “Spot-Fix and Lock” Strategy

In other words, fix the most critical data issues in high-priority areas and immediately lock processes to prevent further degradation.

Start by identifying high-impact errors (eg, duplicates in customer records, incorrect pricing). Once you identified the set(s), correct these errors manually or via scripts. Finally, implement basic process locks, such as requiring specific fields to be filled before records are saved or restricting edits to validated data.

You end up with stabilised data quality in key areas, reducing downstream chaos.

Once the immediate chaos is controlled, start laying the groundwork for systematic improvements and building a foundation for sustainable data management. For instance, create a roadmap for addressing root causes (eg, better governance, new necessary tools). But whatever you do, don’t forget to document lessons learned from the crisis to guide future processes.

The key principle here is: stabilise, not perfect.

Remember, your goal is to bring enough order to stabilise operations and decision-making, even by using imperfect solutions. Once the immediate crisis is averted, you can gradually transition to proactive long-term strategies.

3. Scalability of Data Infrastructure

Let’s see what we can do with infrastructure bottlenecks caused by over-relying on basic tools that now can’t handle the exponential growth of data as the organisation scales. Instead of smooth operations, we have slow analytics processes, delayed insights and increased costs because systems struggle to keep up. 

Again, on a normal day, you would simply:

• Adopt cloud-based, scalable data storage and processing solutions.
• Use modular systems that can grow with the organisation.
• Plan for scalability when designing data architectures.

But that simply isn’t the case. Your predecessors (if any), didn’t quite do the job right and now you have a serious problem – unscalable data in a fast-growing company.

When faced with such an infrastructure in a rapidly growing organisation without the resources to invest in modern solutions, you must focus on triage, optimisation and tactical solutions. The goal is to stabilise the infrastructure to support growth in the short term while preparing for future scalability once resources are available.

3.1. Triage the Infrastructure Bottlenecks

Your priority is identifying the most critical bottlenecks in the current infrastructure that directly impact operations or decision-making. That is, perform a rapid audit of the existing infrastructure to identify pain points (eg, slow query response times, system outages, capacity issues). 

Once identified, prioritise fixing the systems that handle mission-critical data (eg, sales, billing, customer support).

This should give you a clearer understanding of where to focus limited resources for maximum impact.

3.2. Optimise Existing Resources

While you are already dealing with bottlenecks, activate the afterburner by squeezing the maximum performance out of the existing infrastructure with targeted optimisations.

For example:

• Database Tuning:
  • Optimise query performance by indexing critical columns, rewriting inefficient queries and archiving old data.
  • Partition large tables if possible to improve performance.
• Storage Management:
  • Compress data to reduce storage requirements.
  • Move cold or historical data to cheaper, offline storage (eg, local hard drives or NAS).
• Batch Processing:
  • Shift non-urgent data processing tasks (eg, report generation) to off-peak hours.

If done correctly, you should see immediate performance improvements without requiring new infrastructure.

3.3. Implement Stopgap Solutions

The play here is to introduce temporary fixes to alleviate pressure while preparing for longer-term improvements. 

Here’s what you can do to achieve this:

• Use local servers or existing hardware more efficiently (eg, repurpose underutilised machines as temporary data nodes).
• Set up lightweight, open-source tools for specific needs (eg, Apache Kafka for message queuing, PostgreSQL for database expansion).
• Leverage basic automation tools to reduce manual intervention in data handling.

These solutions may appear trivial but keep in mind what we are trying to achieve here and under which circumstances. We ultimately want stabilised infrastructure to support ongoing growth, even if suboptimal.

3.4. Segment and Prioritise Data Loads

Data don’t need to be processed or stored at the same priority level. Therefore, segregate data workloads based on their importance and urgency. For example:

• Categorise data into tiers (critical, operational, historical).
• Allocate the best resources to the most critical data sets.
• Limit real-time processing to essential data and defer non-critical processing.

The cumulative effect is reduced strain on the infrastructure without sacrificing business-critical operations.

3.5. Leverage Community and Open-Source Resources

Sometimes, you don’t have any other choice but to enter the dark ally of open-source tools and use them to address specific pain points in the data infrastructure. 

Use open-source tools like MySQL, PostgreSQL or SQLite for additional database capacity and implement lightweight ETL solutions like Apache NiFi or Singer for data integration. Finally, make sure to monitor system health with, for example, Zabbix or Prometheus.

None of us prefer open-source solutions, but they are cost-effective and scalable enhancements. For instance, we are utilising Mautic as our central nervous system and a single source of truth. Our CTO, Jason Noble, spent a lot of sleepless nights getting that open-source beast to life and keeping it updated. However, it was worth it. We don’t spend thousands on monthly subscriptions and we alone own all data. Would it be the same if we had chosen HubSpot, for example, that’s highly questionable. 

3.6. Build Manual Processes as Interim Solutions

When automation or scaling proves impractical for any number of reasons, use manual processes to handle critical data workflows. 

You simply assign dedicated teams or individuals to manage data flows that the current infrastructure cannot handle (eg, manually consolidating reports or transferring data between systems). Just remember to use templates or scripts to streamline repetitive tasks.

It’s not exactly practical and can cause delays, but these short-term solutions keep the business running without overwhelming the infrastructure.

The key principle here is: survival first, perfection later.

In this critical phase, focus on stabilising the infrastructure and ensuring business continuity. While the current environment may remain suboptimal, these actions will buy you time to secure the resources and strategic alignment necessary for sustainable, long-term growth.

And remember, no matter the situation, begin laying the groundwork for scalable solutions even if resources are tight. Begin consolidating fragmented systems into a single source of truth wherever feasible. Also, document the current infrastructure and create a lightweight plan for migration to a scalable architecture once resources become available. And in that little spare time you get around lunch, try to identify low-cost, incremental investments that could ease scalability bottlenecks.

4. Talent Shortages and Skill Gaps

Start-ups often struggle to attract and retain skilled data professionals due to competition from larger organisations. That lack of expertise can result in underutilised data assets and suboptimal decision-making.

Commonly, a CTO would deploy these three strategies:

• Upskilling existing team members in data literacy and analytics.
• Partnering with external consultants or leveraging outsourcing for specialised needs.
• Cultivating an attractive work culture to retain data talent.

Now imagine the scenario in which none of the proposed mitigation strategies works, at least not in the long run because the small team of only a few simply can’t find additional time to upskill in data literacy and analytics (they are software engineers). Partnering with external consultants or some extensive outsourcing is out of the question and the work atmosphere is so grim that it is impossible to create and cultivate an attractive work culture to retain data talent. But the paycheck on the other hand is so big that you don’t want to quit and search for something else. What can you do?

Here is the list of the most realistic strategies:

1. Identify the smallest set of tasks that deliver the most significant results and focus only on those.
2. Use simple, low-code/no-code automation to reduce repetitive work and free up time for the team.
3. Empower non-technical staff to handle basic data-related tasks with user-friendly tools.
4. Accept that the data infrastructure and processes won’t be perfect and focus on “good enough” solutions.
5. Create opportunities for your team to learn informally and in small increments, without requiring extensive upskilling efforts.
6. Collaborate with other departments to share responsibilities or gain access to additional skills.
7. Improve communication about current constraints and challenges to align expectations.
8. If possible, bring in limited short-term help from freelancers or contractors for specific tasks.
9. Implement changes that yield long-term benefits without requiring ongoing maintenance.
10. Even in a grim atmosphere, recognise and reward your team’s efforts to boost morale.

As you can see, the guiding principle here is: stabilise to survive. In other words, if you are in a highly stressful and negative environment with limited resources and a small overburdened team, just focus on stabilising the situation and delivering “good enough” results. 

Therefore, prioritise ruthlessly, automate strategically and leverage creatively to ensure the team survives the current challenges while laying the groundwork for future improvements.

5. Balancing Speed with Rigour

As we said early on, start-ups and fast-growing organisations are often forced to make quick decisions to maintain their competitive edge. This leads to shortcuts in data analysis or overreliance on intuition.

Normally, a technology leader would implement these three strategies to balance speed with rigour:

• Create streamlined yet robust processes for data validation and analysis.
• Foster a balance between agility and thoroughness in decision-making.
• Encourage cross-functional collaboration to validate insights before acting.

But what happens when data silos hinder speed and rigour while pressure for speed amplifies silos? 

Let’s use case studies to better understand this causal relationship: 

• Scenario 1: A start-up rushes to launch a new product. Sales and marketing teams use different platforms to track leads and engagement. Decisions about the product’s target audience are made based on siloed data, leading to misaligned messaging and wasted resources.
• Scenario 2: A scale-up prioritises speed in reporting but lacks a unified data warehouse. Analysts spend time manually consolidating data, delaying insights and increasing the risk of errors, which undermines rigour.

How to break this vicious cycle?

In ideal circumstances, organisations would employ the following strategies:

• Adopt centralised data platforms or warehouses early on to enable seamless access across teams.
• Encourage teams to adopt scalable systems even if they take longer to implement initially.
• Establish cross-functional practices by facilitating data sharing and strategic alignment between teams.

Only, we are not that lucky. There are no warehouses, teams still work on legacy (read: rigid and fixed-capacity) systems and nobody shares anything. It even seems that teams pursue different strategic goals. That’s the situation we met after accepting the role. 

What we need now is a phased, tactical approach that delivers quick wins while laying the groundwork for broader transformation. It is essentially a five-step strategy:

Step 1: Triage and Stabilisation

In this step, our priority is to identify critical interdependencies so we can get some clarity on immediate priorities to stabilise the situation.

To find out, we can conduct a rapid assessment of the most critical pain points. For example:

• Which decisions are being delayed or compromised due to silos?
• What strategic misalignments are most damaging to the company?

Then, we need to focus on cross-functional bottlenecks where silos directly affect speed and rigour. This requires the creation of a temporary “Data Task Force” or a small agile cross-functional group that will address critical silos by accessing and consolidating data needed for immediate priorities. The good practice here is to assign members from key teams (eg, product, finance, operations) to represent diverse perspectives.

Eventually, all these efforts should create a temporary workaround that will enable collaboration and quick fixes.

Step 2: Quick Wins to Build Momentum

Start by creating a “Minimum Viable Integration” to achieve basic data sharing without major resource investments. That is, use lightweight solutions to connect siloed systems, focus on critical data flows and automate repetitive processes.

Next, establish a “Single Source of Truth” for critical metrics to enable shared visibility into business performance, fostering alignment.

Finally, pilot cross-functional decision reviews for high-stakes decisions to create a foundation for a gradual cultural shift toward collaboration and shared accountability.

Step 3: Establishing a Foundation for Change

To reduce strategic misalignment and increase clarity, teams must unify under the same goal framework. To get there, team leads need to be aligned on well-defined company-wide strategic goals. These goals must then be broken into measurable objectives tied to specific team deliverables.

It’s only now that you can start prioritising tactical investments in scalability by implementing high-impact, low-cost upgrades to legacy systems (eg, replacing outdated software with lightweight cloud-based tools). 

You can easily justify these investments by linking them to business outcomes like faster time-to-market or improved customer satisfaction. Just remember to start small to fit within resource constraints.

The outcome is gradual modernisation without overwhelming the organisation.

Step 4: Cultural and Process Transformation

You want to achieve three goals here:

1. Incentivise data sharing to reduce resistance to collaboration and improve data flow.
2. Simplify and streamline processes to improve operational efficiency without introducing unnecessary complexity.
3. Drive a mindset shift (lead by example).

Step 5: Measure and Adjust

What to track and measure? 

Well, track key indicators such as decision turnaround times, collaboration frequency and strategic goal alignment. Use these metrics to gauge the effectiveness of your interventions. Just remember to regularly share progress updates with leadership and the broader team.

How to adapt for scaling?

• Build on early successes to expand collaboration and data-sharing practices.
• Gradually phase out legacy systems, reinvesting savings into more scalable solutions.
• Adjust priorities based on the evolving needs of the organisation.

The result is sustained momentum and long-term scalability.

Conclusion

In challenging environments, maintaining data integrity for strategic planning requires a balance between stabilising immediate risks and building a scalable foundation for the future. Quick wins, collaboration and adaptability are essential to breaking the cycle of dysfunction and driving sustained organisational success.

The key takeaways:

1. Understand and prioritise immediate risks.
2. Establish quick, practical solutions.
3. Promote collaboration and alignment.
4. Balance speed with rigour.
5. Leverage existing resources creatively.
6. Drive cultural transformation.
7. Measure progress and adapt.

Through four weeks and sixteen lectures in Module 8 of our Digital MBA for Technology Leaders, the faculty of senior executives responsible for data management in their organisations, teach this and other subjects in much more detail, using years-long experience. You will learn how to adjust to an array of different circumstances to, ultimately, maintain data integrity even in worst-case scenarios.