Whenever they must manage distributed teams, technology leaders face the dual challenge. The first is driving outcomes. The second is nurturing a coherent, connected culture. This playbook offers a step-by-step guide to building resilient remote engineering teams equipped to thrive across time zones and communication barriers.
1. Assess Current Team Structure
Start with a situational audit:
Map out current roles, overlap in working hours, and collaboration effectiveness.
Identify gaps in visibility, autonomy, and performance tracking that might hinder remote efficiency.
2. Define Remote Work Policies
Establish policies that align with your business objectives and team diversity. Include:
Expectations for availability
Documentation standards
Meeting etiquette
Boundaries between work and rest.
3. Set Up Communication Cadence
Regular touchpoints are essential. This is what you should do:
Use daily stand-ups, weekly retrospectives, and monthly strategy calls to synchronize efforts.
Tailor formats to avoid fatigue and ensure inclusion across time zones.
4. Implement Collaboration Tools
Select and integrate a tech stack for seamless collaboration. Essentials include:
Version control (Git)
Project tracking (Jira)
Documentation (Confluence)
Messaging (Slack).
TIP: Automation can bridge tool silos.
5. Onboard Remote-Focused Culture
Onboarding should instill values, not just workflows. Therefore, introduce peer mentoring, asynchronous onboarding journeys, and culture-building rituals like virtual coffee hours to embed a shared ethos.
Use dashboards for velocity, PR cycle time, and DORA metrics.
Supplement with pulse surveys and regular 1:1s to uncover sentiment trends.
7. Create a Continuous Feedback Loop
Enable retrospectives every 4–6 weeks.
Capture feedback anonymously and publicly.
Adapt rituals and tooling based on evolving needs to foster continuous improvement.
With intentional leadership and robust systems, remote engineering teams can exceed the impact of colocated peers. Remember, resilience emerges not from tools but from trust, clarity, and shared purpose.
Innovation in the tech industry extends far beyond the creation of new gadgets and software; it encompasses novel approaches to operations, strategy, and organisational structure. True innovation involves rethinking how things are done, aiming for efficiency and effectiveness in achieving or creating outcomes.
For example, the shift from traditional agile methods to more holistic approaches, like those discussed in “Team Topologies,” illustrates innovation in workflow organisation. This model focuses on streamlining communication and collaboration within development teams to enhance productivity and software delivery.
An insight from the late Steve Jobs, “Innovation distinguishes between a leader and a follower,” captures the essence of this broader view of innovation. It’s not just about new products but about pioneering new ways to approach business challenges and market needs.
Innovative leaders drive their companies to adopt such forward-thinking strategies, fundamentally altering the competitive landscape and setting new standards for their industries.
Innovative leadership means cultivating an environment where every operational process is questioned, and improvements are continually sought.
Leaders like this inspire their teams to think beyond conventional boundaries, whether in developing new technologies or in devising better ways to work together. In other words, they understand that innovation is about effective execution as much as it is about brilliant ideas, transforming their products and processes to achieve remarkable results.
In the fluid and constantly changing world of tech, decisiveness is crucial, particularly in agile teams that often operate with a flat hierarchy and are expected to be autonomous. However, this structure can sometimes lead to a scenario where “too many chefs and not enough cooks” impede swift decision-making.
Being “Decisive” means cutting through the potential gridlock of collaborative environments to make timely, effective decisions based on data and the opinion of a variety of team members.
For example, in many tech companies, while the agile methodology promotes team autonomy, it can also lead to delays if not managed with decisive leadership.
A decisive leader in this context would ensure that while collaborative input is essential, there are clear protocols and appointed decision-makers at critical junctures to prevent stagnation and ensure that decisions are made efficiently and effectively. These decisions should ideally avoid “trap-door” decisions, where the stakes are high and the path back is not feasible. Leaders need to approach such irreversible decisions with great care and strategic foresight.
Theodore Roosevelt’s insight reflects the essence of such leadership: “In any moment of decision, the best thing you can do is the right thing, the next best thing is the wrong thing, and the worst thing you can do is nothing.” This highlights the importance of decisiveness, suggesting that the risk of making a wrong decision is often more favourable than the cost of inaction.
Decisive leaders establish frameworks where decisions are data-informed yet swift, enabling their teams to respond quickly to changing market demands and technological advances. This not only accelerates innovation but also instills a culture of trust and accountability, essential in maintaining the momentum in fast-paced tech environments.
Adaptability in tech leadership is the ability to change one’s approach and strategy to meet new challenges and environments effectively. It is a trait that allows leaders not just to respond to changes but to thrive amid them.
Bruce Lee’s famous philosophy, “Be like water“, perfectly encapsulates this idea. Water is flexible; it flows into any shape and can either gently fill gaps or crash with mighty force. Similarly, adaptive leaders can navigate the complexities of the tech industry, adjusting their strategies and methodologies as needed without losing momentum or focus.
This adaptability is crucial for technology innovation. The tech landscape is constantly evolving with new challenges emerging at a rapid pace. Leaders who embrace a flexible mindset can foresee potential changes and pivot their operations or product development processes accordingly. They foster environments where teams are encouraged to experiment and learn from each outcome, whether it’s a success or a setback.
Incorporating adaptability into leadership involves continuous learning and an openness to feedback. It means staying updated with the latest technological advancements and industry trends, and being ready to overhaul outdated practices that no longer serve the evolving market demands. Thus, being like water isn’t just about being flexible—it’s about being powerfully responsive and resilient in the face of change.
Recently, we surveyed technology leaders and engineers to explore the key difficulties organisations face regarding contemporary work arrangements. The study presented participants with a list of potential challenges and recorded the frequency with which each was identified.
As you can see, sustaining creativity and innovation was most frequently selected, closely followed by fostering a sense of belonging.
Maintaining effective collaboration, onboarding new talent, balancing flexibility with cohesion and preventing work culture fragmentation are also high up in the focus.
In contrast, addressing overwork and optimising office space were noted far less often. Nonetheless, to some, they are pivotal.
This article provides immediately applicable solutions to these challenges, offering technology leaders the practical tools to navigate workplace complexities effectively.
1. Sustaining Creativity and Innovation
Challenge: In a hybrid or remote environment, fostering creativity and innovation can be difficult due to reduced spontaneous interactions and isolated workflows.
In 1968, Dr. Spencer Silver, a chemist at 3M, was attempting to create a super-strong adhesive for use in aircraft construction. Instead, he accidentally created a weak, pressure-sensitive adhesive that could be peeled away easily without leaving a residue. At first, this invention seemed like a failure – after all, who would want a glue that doesn’t stick properly?
It wasn’t until 1974 that his colleague, Art Fry, came up with the idea of using the adhesive to create bookmarks that wouldn’t fall out of his hymnal. This spark of creativity led to the development of the Post-it Note.
Would Fry even find out about Silver’s invention if they worked remotely and had isolated workflows?
Yes, the first action step is to synchronise workflows. In-house, hybrid or fully remote; it doesn’t matter as long as the workflows are synced.
But there’s another problem – it’s not always people’s fault they are not creative and innovative.
Get ready to face the hard truth about your leadership style and/or organisation’s culture in general.
Only 24% of employees reported feeling curious in their jobs regularly.
Approximately 70% said they face barriers to asking more questions at work.
In other words, leaders might believe they allow creativity and entice curiosity, but their team members certainly don’t feel that way.
This leads us to the most interesting thing in the study and that’s the list of 3 key innovation/creativity blockers:
Leaders often believe that encouraging curiosity will lead to a costly mess and make the company harder to manage.
There’s a concern that allowing employees to explore their interests would lead to more disagreements and slow down decision-making processes.
Despite listing creativity as a goal, people frequently reject creative ideas when actually presented with them.
Hence, the
Immediate Solutions
There are a few important takeaways from this study and general practice that should mitigate the problem of lack of creativity and innovation:
Create an environment where employees feel safe to ask questions, explore and share new ideas without fear of ridicule or punishment.
Understand that exploration doesn’t always produce immediately useful information but often yields better long-term solutions that require many mini-solutions in between to work.
While some structure is necessary, allow for flexibility in exploring new ideas and approaches.
Implement recognition systems that reward not just successful outcomes, but also the process of exploration and learning.
However, not all people are creative (and innovative) by default. In fact, only a handful are. So what is the simplest method to entice creativity in otherwise uncreative individuals?
This may sound counterintuitive, but setting strict limits can be your ace in the hole.
Constraints, when approached with the right mindset, can be a catalyst for creativity rather than an obstacle.
In 1960, Bennett Cerf, the founder of Random House, bet the famous Dr. Seuss $50 that he couldn’t write an entertaining children’s book using only 50 unique words. This challenge came after Dr. Seuss had already successfully written “The Cat in the Hat” using a limited vocabulary of 236 words from a list of 348 words that first-graders should know.
The result?
Dr. Seuss accepted the challenge and produced “Green Eggs and Ham,” which became:
His best-selling book
The fourth best-selling children’s hardcover book of all time
A book that has sold over 200 million copies worldwide
Creativity thrives under limitations: Rather than stifling creativity, the 50-word constraint forced Dr. Seuss to be more innovative in his storytelling.
Quality over quantity: Despite the limited vocabulary, the book became a masterpiece of children’s literature.
Problem-solving skills: The constraint required Dr. Seuss to approach the writing process differently, enhancing his problem-solving abilities.
Focus and efficiency: The word limit forced Dr. Seuss to be concise and focused in his storytelling.
In business, limited resources often lead to innovative solutions while time constraints can increase productivity and focus.
2. Fostering a Strong Sense of Belonging
Challenge: Employees, especially those working remotely, often struggle with feeling disconnected from the company culture and their colleagues.
Take it from someone who’s been working remotely for the last 11 years – this is a tough nugget to break. It will take a hard personal investment to create a sense of belonging. Nonetheless, it is achievable.
Now, the widely used solutions are:
Virtual Team-building Activities
Regular Check-ins
Weekly Team Meetings
Monthly Team Meetings
Recognition Programs
Company-wide Rituals
Make no mistake; all of them work, but only if you a) give a team member a true sense of purpose and b) hold them accountable.
Consider a complex, interconnected machine – a sophisticated network of gears, levers and circuits. This machine, let’s call it “Synergy”, represents a tech organisation operating in a remote environment. Its smooth functioning depends entirely on the coordinated effort of its individual components – the remote tech teams or members.
The organisation’s leaders, the “Master Engineers”, understand that simply providing the blueprints isn’t enough. They need to instil a sense of purpose and accountability within each team, each member, each cog in the Synergy machine. It is an eight-step process.
Step 1: Ensuring Seamless Operation
To cultivate purpose, the Master Engineers ensure every team member understands their critical role in the machine’s overall operation. They explain how even the smallest line of code contributes to the larger function, and how each bug fix prevents a catastrophic system failure.
Step 2: Directing Personal Growth into Synergy’s Progress
They also work with each team to set “precision-engineered” goals – targets that align not only with Synergy’s overall performance metrics but also with individual career trajectories. A junior developer might aim to master a new coding language, while a senior architect might focus on designing a more efficient data flow.
Step 3: Reinforcing the Significance of Individual Contributions
Regularly, the Master Engineers showcase the impact of each team’s work – how a new feature improved user experience and how a security patch prevented a data breach. In other words, they demonstrate real-world consequences.
Step 4: Laying the Groundwork for Responsibility
To enhance accountability, the Master Engineers establish clear “performance parameters” – specific, measurable outcomes for each team and project. These defined expectations might include code quality metrics, sprint completion rates or client satisfaction scores.
Step 5: Encouraging Self-regulation and Peer Oversight
Synergy is equipped with a sophisticated “monitoring system” – a suite of project management tools and performance dashboards that provide real-time visibility into the machine’s operation.
Step 6: Ensuring Alignment
The Master Engineers also conduct regular “calibration sessions” – one-on-one and team meetings to discuss progress, address challenges and performance deviations and refine goals.
Step 7: Fostering a Culture of Ownership
They empower team members to take initiative in problem-solving and decision-making.
(QUOTE)When individuals feel responsible for their part of the machine, they’re more likely to hold themselves accountable for the results.
Step 8: Reinforcing the Value of Accountability
Finally, the Master Engineers recognise and reward exceptional performance – publicly acknowledging teams and members that consistently exceed expectations and demonstrate strong ownership.
The outcome?
An environment where each team member feels deeply connected to their purpose and takes genuine ownership of their responsibilities. Such a combination not only drives performance but also ensures long-term stability and success.
3. Maintaining Effective Collaboration
Challenge: Coordinating efforts across distributed teams can result in inefficiencies, miscommunications and delays.
The problem is that managers stick to the paradigm of the office environment, completely ignoring the fact that their teams operate from living rooms, bedrooms and basements of their homes. And homes have an opposite paradigm.
Traditional office environments are structured for face-to-face interactions, spontaneous conversations and immediate feedback. Homes, on the other hand, are designed for personal life and privacy, creating a fundamental paradigm shift.
Moreover, office environments rely heavily on non-verbal cues, body language and spontaneous exchanges, which are largely absent in remote settings.
Immediate Solutions:
When using collab platforms like Nifty, enforce ‘tasking’ instead of messaging for even the smallest and simplest of tasks – without exceptions. In other words, the “Can you check that update” message transforms into the “Check the XYZ Update” task.
Ensure clarity in task ownership and deadlines.
Create and enforce the use of priority tags in tasks (eg, High Priority, Medium Priority, Low Priority)
Maintain an immutable team meeting schedule (to create a sense of expectation and, eventually, a loop of habit).
Set clear agendas and outcomes for every type of meeting.
During meetings, entice transparency and proactive updates to avoid misunderstandings.
Create a centralised knowledge base by developing and maintaining a repository of processes, procedures and frequently used resources.
Whenever possible, use visual project roadmaps that detail timelines, milestones and responsibilities.
4. Onboarding and Developing New Talent
Challenge: Remote and hybrid setups make it harder for new employees to integrate quickly and build meaningful workplace relationships.
If you can put together an extremely precise onboarding program, even better.
That’s basically all you can do besides providing access to the centralised knowledge base and training programs. If it sticks, fine. If not, repost the job ad. Some people are simply not a team material and there’s nothing you can do about it.
Here’s the thing. Onboarding and developing a new talent is directly related to and dependent on a sense of belonging. So by focusing on #2 (Fostering a Sense of Belonging), you will effectively address #4 (Onboarding).
Belonging is a key factor because a sense of belonging is crucial for new employees to feel engaged, confident and committed to their organisation.
5. Balancing Flexibility with Organisational Cohesion
Challenge: The push for flexible work arrangements can sometimes undermine organisational unity and alignment with company goals.
We are back to the shifting paradigm where an employee constantly changes between home and office. The first thing you need to do is to create a shared digital workspace – and use it. It doesn’t matter if a part of the team is in the office; they still must use that workspace as long as even a single member is remote.
What happens is that team members who work from the office, tend to disregard the fact that some of them are missing. In their minds, that person is on leave or has a day off. It’s simply part of that common office paradigm we mentioned earlier.
Additional Solutions:
Define clear policies (guidelines) that balance autonomy with necessary in-office collaboration.
Establish overlapping work hours for synchronous communication.
If in any way possible, schedule physical meet-ups to reinforce team spirit.
Create an internal knowledge base to ensure consistency in workflows and processes.
Prioritise accountability by all means.
If you manage distributed teams, involve all team members in defining and refining the team’s vision and goals.
6. Preventing Fragmentation of Work Culture
Challenge: A dispersed workforce can lead to fragmented cultures, where different teams develop disconnected subcultures.
Immediate Solutions
Several experienced leaders have successfully addressed this issue through innovative approaches and strategic initiatives:
Automatic Created a Unified Digital Culture
Automattic, the parent company of WordPress.com, provides an efficient example of maintaining a cohesive culture in a fully distributed workforce. Their leadership team adopted a “write early, write often” strategy to foster transparency and collaboration.
By encouraging team members to share updates and feedback in shared documents, they created a virtual environment that mimics the organic interactions of a physical office.
IBM’s leadership team faced similar challenges when shifting to a remote-first strategy in 2020. Initially, they observed a major decrease in employee engagement compared to in-person work. To combat this, IBM’s leaders implemented three innovative solutions:
Regular virtual town halls to maintain open communication.
Social hours to foster informal connections.
Integration of virtual reality (VR) in leadership training programs.
Tariq, a young leader in a global firm, successfully addressed cultural fragmentation in his 68-person division spanning 27 countries and 18 languages. His approach included:
Introducing a unifying team motto: “We are different yet one”.
Creating opportunities for employees to share their cultures.
Implementing a zero-tolerance policy for cultural insensitivity.
These initiatives helped bridge cultural divides and rebuild team cohesion, demonstrating the importance of acknowledging and celebrating diversity while fostering unity.
We Emphasise Shared Purpose
CTO Academy leaders consistently remind team members of their common purpose and how their work contributes to overall company goals. During weekly team calls, for instance, a CEO reviews the group’s performance relative to company objectives. This practice helps maintain focus and unity, especially when team members are geographically dispersed like we are.
Personal Connection and Recognition
A manager based in Dallas, Texas, inherited a large team in India following an acquisition. To prevent cultural fragmentation, he:
Involved remote employees in important decisions.
Maintained frequent contact to discuss ongoing projects.
Personally called team members to give them their birthdays off.
Leverage technology to improve communication and capitalise on diversity while emphasising unity.
Maintain personal connections across geographical boundaries.
7. Addressing Employee Overwork and Burnout
Challenge: The blurred boundaries between work and home life in remote and hybrid setups have led to increased burnout rates.
If this is the prevalent issue in your organisation, you need to take a big step back and reorganise your processes. There is something in your operations that disturbs the balance.
Immediate Solutions:
1. Set Boundaries on After-hours Communication
In Germany, for example, contacting employees after hours is generally prohibited, with exceptions for emergencies and specific roles. The aim of this measure is to:
Actively promoting and tracking employee vacations to ensure they take breaks can be somewhat challenging when managing distributed teams. In our experience, the best approach is to utilise your central digital workspace.
Our COO, for instance, has implemented a time-off calendar where team members easily schedule their time off in Nifty after manager approval. This provides immediate visibility for the entire team, showing who’s out and for how long. It simplifies tracking and helps with processes because a team member can’t receive a task with a deadline that doesn’t take the time off into account.
3. Implement No-meeting Days
These are basically dedicated days for deep work without interruptions.
4. Provide Mental Health Resources
In a fast-paced environment, burnout is inevitable. One approach is to provide comprehensive mental health resources such as counselling services, webinars or self-help materials. They should be:
Easily accessible
Encouraged
Utilised
Confidential
This will enable you to identify and manage stress and burnout early on.
8. Optimising and Adapting Office Space
Challenge: With hybrid work models, many companies struggle to justify office expenses while ensuring the space remains functional.
Real-time desk and room booking systems to prevent scheduling conflicts and maximize space usage.
Occupancy sensors to provide data on space utilisation, helping manage office density and optimise layouts.
All-in-one platforms like Microsoft Places and Gable for comprehensive workspace management, offering features such as AI-driven work schedule optimisation and access to flexible workspaces.
The point is to rethink the organisation of the office space because clearly something is off. Maybe you have collaboration hubs but no quiet thinking bunkers where an employee can retreat to contemplate the problem without distractions. Perhaps it’s overcrowded or oversaturated with unnecessary equipment and simple re-arrangement could go a long way.
This is where technology or a good old professional interior designer helps.
Now, you may have also heard of ‘hot desking’ otherwise known as ‘hoteling’. If you are thinking about implementing such an option, consider these factors:
Loss of personalisation
Psychological discomfort
Reduced productivity
Weakened social structures
Decreased job satisfaction
Sense of belonging
That’s the less discussed outcome of booking desks and spaces practice. As a species, we are wired to grow attached to our personal spaces and the office desk is no exception. Drop us anywhere and we’ll transform a hole in a rock into a cosy and warm place with a personal signature.
Just for fun, imagine a bunch of kids storming into a room filled with cool toys – day after day.
Conclusion
So to sum up:
Creativity and innovation can be fostered through synchronised workflows, a safe environment for exploration and perhaps setting strict limits to encourage creative problem-solving.
Giving team members a true sense of purpose and holding them accountable can help them feel strongly connected.
Effective collaboration can be maintained by enforcing tasking instead of messaging, ensuring clarity in task ownership and deadlines and creating a centralised knowledge base.
Onboarding and developing new talent can be effectively addressed by focusing on fostering a sense of belonging.
Balancing flexibility with organisational cohesion requires creating a shared digital workspace, defining clear policies and establishing overlapping work hours for synchronous communication.
Leveraging technology, maintaining personal connections across geographical boundaries, and emphasising shared purpose can prevent the fragmentation of work culture.
Addressing employee overwork and burnout requires setting boundaries on after-hours communication, encouraging time off, implementing no-meeting days and providing mental health resources.
Finally, optimising and adapting office space can be done by using smart space management systems and rethinking the organisation of the office space.
Artificial Intelligence is evolving beyond narrow, task-specific applications into agentic AI—systems capable of making autonomous decisions, adapting to dynamic environments and taking independent actions to achieve goals. This paradigm shift presents unprecedented opportunities for automation, efficiency and innovation. However, as organisations move toward deploying AI agents in critical operations, technology leaders must address several fundamental concerns.
For CTOs and tech executives in general, the question is no longer whether to implement agentic AI but how to do so responsibly and securely. The risks of unchecked autonomy, biased decision-making and unpredictable behaviour demand a structured approach to AI governance, validation and human oversight.
This article explores the core challenges of agentic AI, backed by real-world case studies, and outlines the best mitigation strategies to ensure safe, accountable and effective AI deployment.
When users share data with AI chatbots, it is stored on the servers of companies like OpenAI, Microsoft and Google—often without a straightforward way to access or delete it. This raises concerns about sensitive information being shared with chatbots like ChatGPT that could unintentionally become accessible to other users.
By default, ChatGPT saves chat history and uses conversations to improve its models. While users can manually disable this feature, it’s unclear whether the setting applies to past conversations retroactively or if it’s working at all because it is virtually impossible to audit data that OpenAI and other providers use to train their models.
Technology leaders face a dilemma here: We either act in good faith and use products or ban the use of Gen AI tools as Samsung did. If we do use those products, we must accept three possibilities:
Employees may input confidential information into AI without realising it could be stored or used for future model training.
Even with data governance policies in place to prevent sensitive data from being shared with external AI services, history taught us that providers often ignore those rules because data is a commodity.
Due to a lack of visibility and access control, a company’s secrets could be exposed without a clear way to delete or retract them.
This is what we can do to at least minimise exposure:
Use role-based access controls (RBAC) to limit data access to only necessary personnel or AI modules.
Implement access controls and encryption at all levels to prevent AI from having unrestricted access to sensitive data.
Instead of centralising all user data, AI can learn from noise-injected distributed datasets without exposing raw information. This prevents raw data exposure but does not affect AI capabilities.
Train AI models in secure environments with masked or anonymised data (synthetic data instead of real user information w/ Zero Trust architectures).
Ensure that AI-driven data processing aligns with compliance requirements (requires AI explainability functionality).
That’s, unfortunately, the reality because we have limited control over data protection when using a third-party SaaS. But what can we do to prevent Agentic AI systems from acting erratically?
2. Loss of Control
Agentic AI systems and AI in general could act unpredictably. Often, this refers to pursuing objectives misaligned with our intentions. This concern is even more emphasised in high-stakes scenarios because we entrust a complex code with the “black box” feature to make decisions on our behalf.
The malfunctioning can cause an array of implications. For example:
Risk of harmful outcomes.
Inability to intervene effectively.
Potential cascading failures.
On March 18, 2018, an Uber self-driving test vehicle in Tempe, Arizona, struck and killed a pedestrian, Elaine Herzberg. This was the first recorded fatality involving a fully autonomous vehicle, raising serious concerns about loss of control in AI-driven systems. The vehicle’s onboard AI was designed to detect and react to obstacles autonomously, but a failure in decision-making and override mechanisms led to a tragic accident.
The AI incorrectly classified the pedestrian as an unknown object rather than a human, delaying its response. To make things worse, Uber had disabled the vehicle’s built-in emergency braking system, relying entirely on AI-driven decision-making. However, the system was tuned to reduce false positives, meaning it hesitated before deciding to stop which turned out to be a fatal miscalculation.
A human safety driver was present but not paying attention at the critical moment, as AI was expected to handle the situation. The software did eventually order the car to brake 1.3 seconds before the collision but it was too late.
This incident just goes to show that blind reliance on Agentic AI — programmed by humans — can have devastating outcomes.
Mitigation Strategies for Loss of Control in Agentic AI
1. Goal Alignment and Robust Objective Design
Ensure AI systems have clearly defined objectives that align with human values and intentions.
Use techniques such as reward modelling to guide the system’s behaviour toward desired outcomes.
Regularly test the system in diverse scenarios to ensure its objectives remain aligned.
A good example is OpenAI’s approach with reinforcement learning from human feedback (RLHF). This method uses active human guidance to shape the system’s behaviour, ensuring that its autonomous decisions align with human intentions.
2. Control Mechanisms and Fail-Safes
Build robust mechanisms for human oversight, such as kill switches, manual overrides or adjustable autonomy levels.
Ensure that all systems have multiple layers of control to ensure humans can intervene and regain control if the AI behaves unexpectedly.
As much as it can be difficult sometimes, following industry standards and regulatory frameworks ensures the safe development and deployment of agentic AI. That said, both developers and end users should continuously work with policymakers and standards organisations to enforce safety protocols and regular audits.
And the prerequisite for that is monitoring and updating; in other words, deploying systems with continuous monitoring capabilities to detect and address deviations from expected behaviour. For example, AWS and Azure allow developers to update and retrain deployed models to maintain performance and control.
3. Ethical and Moral Challenges
Agentic AI systems face ethical dilemmas, such as deciding whose safety to prioritise or whether to follow instructions that conflict with moral principles. Decisions may not align with societal values, leading to public backlash or regulatory scrutiny.
To mitigate this, Facebook implemented fact-checking partnerships with third-party organisations to address misinformation and started conducting regular ethical reviews to identify and mitigate unintended harms. Additional tools were developed to prioritise high-quality information and limit the spread of harmful content.
Mitigation Strategies
1. Embedding Ethical Frameworks
Google’s AI Principles explicitly prohibit building AI systems that cause harm or reinforce bias, ensuring ethical guardrails. They collaborated with ethicists, domain experts and diverse stakeholders to define moral principles and embed them into the AI’s decision-making algorithms.
2. Value Alignment through Human-Centric Design
As we already said, OpenAI employed RLHF for ChatGPT, which involves training the model to align its responses with user-defined ethical standards. It is a proven approach to ensure AI systems reflect human values. It is done through regular feedback from diverse groups of users because it’s imperative to have an AI system that reflects a broad range of perspectives.
3. Ethical Audits and Impact Assessments
Microsoft’s AI, Ethics, and Effects in Engineering and Research (Aether) committee regularly reviews the company’s AI projects for ethical risks. The committee conducts regular ethical audits andAI impact assessments (AIIAs) to evaluate the social, environmental and moral implications of AI deployments. This is the practice that can be utilised by every organisation simply by establishing independent review boards to assess ethical risks and provide actionable recommendations.
4. Bias Mitigation
Already mentioned IBM’s Watson Health faced criticism for recommending different cancer treatments based on biased training data. The company addressed this by revising datasets and involving clinicians in the training process.In other words, to eliminate bias from the algorithms:
Autonomous vehicle companies like Waymo conduct ethical scenario testing to evaluate how their systems handle life-critical situations, that is, whom to prioritise in a potential collision. They do that in simulated environments to explore how they respond to ethical dilemmas before deployment. These simulations mimic real-world ethical conflicts and analyse the system’s decision-making process.
4. Security Risks
Agentic AI systems can be manipulated, hacked or even weaponised, with autonomous decision-making amplifying their destructive potential. We all saw that ChatGPT-powered gun on YouTube, didn’t we?
In 2020, the SolarWinds cyberattack demonstrated the risks associated with compromised AI supply chains. Malicious actors injected malware into the Orion software platform, impacting thousands of clients, including government agencies.
This case demonstrated a serious lack of robust monitoring in the software update process and insufficient measures to detect and prevent supply chain attacks. To mitigate this and reestablish trust, the company had to implement code-signing practices and enhanced monitoring tools while partnering with security agencies and third-party audits.
Mitigation Strategies for Security Risks in Agentic AI
OpenAI adopted secure development practices to minimise risks in GPT-based models, including API rate-limiting to prevent misuse. They employ techniques such as differential privacy and secure multiparty computation to protect sensitive data used in AI training and deployment.
3. Adversarial Testing
Tesla tests its autonomous vehicle systems against adversarial inputs, such as altered road signs, to ensure the AI behaves correctly in manipulated environments. They use adversarial examples to evaluate how the system reacts to maliciously crafted inputs. These simulations of real-world attacks have two goals:
Test the AI system’s resilience.
Identify vulnerabilities.
4. Continuous Monitoring and Incident Response
By default, AI systems should integrate robust monitoring and alert mechanisms, enabling swift responses to potential security threats. They detect anomalies and security breaches that are sent to dedicated incident response teams that utilise protocols to address security incidents as they occur.
5. Multi-Factor Authentication (MFA) and Access Controls
Back to basic cybersecurity – limit access to AI systems and their underlying infrastructure using strong authentication methods and role-based access controls. Zero-trust policies are still the best first line of defence.
The additional mitigation strategies are:
Encryption and Data Protection
Collaboration with Security Experts
Regulatory Compliance
5. Accountability and Transparency
It’s often difficult to understand or explain the decisions made by complex AI systems, creating a “black box” problem. This causes challenges in assigning responsibility for errors or harm and complicates regulatory compliance and legal proceedings.
Judges and lawyers could not understand how COMPAS reached its conclusions.
The AI disproportionately predicted higher recidivism rates for Black defendants.
The system operated as a “black box,” with no independent review.
Based on this case, AI models in legal decision-making now require:
Transparent documentation,
AI tools used in courts must pass fairness assessments before deployment and
Most importantly, many jurisdictions banned fully automated risk assessments without human review.
So by implementing explainability, auditing, human oversight, regulatory compliance and stakeholder engagement, AI systems can become more accountable and transparent.
Recommended Tools, Techniques, Practices and Frameworks for Improved Accountability and Transparency of Agentic AI Solutions
Use model-agnostic techniques like LIME (Local Interpretable Model-Agnostic Explanations) or SHAP (Shapley Additive Explanations) to provide insights into AI decisions.
Publish transparency reports about how AI models impact users.
6. Dependence and Over-Reliance
Tesla’s Autopilot system, an advanced driver-assistance AI, has been involved in multiple fatal accidents where drivers over-relied on AI and disengaged from driving responsibilities. Despite the manufacturer’s warning, drivers believed the system was fully autonomous and even ignored alerts prompting them to keep their hands on the wheel.
The problem was that the Autopilot did not always escalate warnings forcefully in the events when drivers became unresponsive.
To solve this issue, Tesla now requires drivers to periodically touch the steering wheel to ensure engagement. The system was also updated to activate more aggressive visual and auditory warnings if the driver fails to take control.
But there is another underlying problem. Over-reliance on agentic AI can lead to the erosion of critical human skills caused by blind trust in automated systems. This can easily lead to system-wide failures when AI malfunctions that can even turn deadly.
AI should assist rather than replace human decision-makers, especially in high-risk sectors. Human operators must maintain their expertise and should not entirely rely on or become dependent on AI. For example, after the Air France Flight 447 crash in 2009, where pilots failed to react properly when autopilot disengaged, airlines introduced mandatory manual flying hours to prevent skill degradation. The same thing could happen to software development and software evolution if we fail to timely address this problem.
To sum up, to prevent dependence and over-reliance on agentic AI, organisations should:
Maintain human oversight and decision authority.
Train workers to retain manual skills.
Implement AI uncertainty indicators.
Create manual override and fail-safe systems.
Use hybrid human-AI decision-making models.
Ensure AI explainability and transparency.
Follow regulatory best practices.
7. Reliability and Accuracy
(click to enlarge/download)
Agentic AI systems may fail to make consistent, accurate decisions in dynamic, uncertain or adversarial environments. Consequently, they may cause catastrophic errors in critical domains.
Regardless, AI-powered chatbots are increasingly used for medical symptom analysis for example. However, AI lacks real-world clinical experience, hallucinates, can fail to identify rare conditions and has no self-checking mechanism. In other words, most LLMs we use daily do not verify their own answers before outputting query results.
Let’s use case studies and real-world examples to see how to improve accuracy so we can rely more on Agentic AI.
Google’s Med-PaLM 2, for instance, initially struggled with accuracy due to biased training data. The company was forced to improve reliability by training on diverse multi-institutional datasets.
Uber’s self-driving car fatally struck a pedestrian in 2018 due to poor real-world validation. Waymo, by contrast, conducted millions of real-world and simulated test miles, reducing failure rates before public deployment. Waymo proved that AI models must undergo rigorous validation and real-world scenario testing before deployment.
IBM Watson for Oncology initially provided incorrect treatment recommendations due to limited training data. The company introduced real-time physician feedback loops, allowing the model to improve through expert corrections. AI could now detect errors and self-correct in real time thanks to feedback loops and improved confidence scoring.
Another way to improve the decision accuracy of Agentic AI is to use multiple AI models. It’s called ensemble learning where multiple models provide independent predictions and vote on final decisions while using backup rule-based systems for high-risk decisions. The best example is NASA’s Mars Rover AI Navigation which uses redundant AI models to cross-validate terrain analysis before making navigation decisions. This prevents mission-critical failures caused by single-model inaccuracies.
Arguably the best approach to developing a reliable and accurate Agentic AI is to force the AI to explain its decisions and flag uncertain predictions for human review. This can be done by incorporating XAI techniques and implementing confidence thresholds that trigger human intervention for low-confidence results. For example, Healthcare AI (DeepMind’s Kidney Disease Prediction) flagged high-risk cases with explainability reports, allowing doctors to verify predictions before acting.
The bottom line is that AI should never operate autonomously in critical situations. In other words, deploy AI as decision support rather than an autonomous agent and mandate manual approval for AI-generated recommendations in high-risk industries. It brings us back to the Boeing 737 MAX MCAS incident where a faulty AI-driven flight stabilisation system overrode pilot inputs, leading to fatal crashes.
The Key Takeaways
To improve reliability and accuracy, organisations should:
Train AI on high-quality unbiased datasets.
Conduct real-world testing and validation.
Implement real-time error detection and self-correction.
Use redundancy (multi-model AI systems) to cross-verify decisions.
Apply explainability techniques (XAI) to flag uncertain predictions.
Ensure regulatory compliance and third-party auditing.
Require human oversight in critical decision-making.
Conclusion
Agentic AI presents immense opportunities but also introduces critical risks such as:
Loss of control
Ethical dilemmas
Security threats
Lack of transparency
Over-reliance
Accuracy failures.
To mitigate these, technology leaders must prioritise human oversight, robust security measures and explainability while enforcing strict governance frameworks.
AI should be an assistive tool, not an autonomous decision-maker in high-risk domains. In other words, human expertise remains central.
Success in deploying agentic AI hinges on continuous validation, adversarial testing, regulatory alignment and adaptive learning models. Organisations that proactively address these challenges will drive trustworthy, resilient and high-impact AI adoption, positioning themselves as industry leaders in safe and scalable AI innovation.
Start-ups and scale-ups often prioritise quick decisions to maintain their competitive edge, which can lead to shortcuts in data analysis or overreliance on intuition. The impact is often immediate because hasty decisions based on incomplete or improperly analysed data can result in missed opportunities or strategic missteps.
This is particularly true when data is fragmented across silos. Teams simply cannot access or integrate information efficiently. This forces tech leaders to either wait for data consolidation (slowing down the process) or make quick decisions based on incomplete data, sacrificing rigour (accuracy).
This article will address these two primary challenges and offer actionable solutions while solving the other three capital problems in data-driven decision-making. However, this is not our normal day at work. In this case, things just cannot be worse. We are operating in a high-pressure scenario where the company is on the brink of financial ruin and you, as a technology leader, inherited a chaotic environment with poor data processes. The goal is to quickly induce enough order to enable survival, even if perfection is impossible.
5 Biggest Challenges for Start-up and Scale-up Tech Leaders in Data-Driven Decision-Making
In any given scenario, the challenges are the same:
Data Silos and Integration
Data Quality and Accuracy
Scalability of Data Infrastructure
Talent Shortages and Skill Gaps
Balancing Speed with Rigour
But, in our situation, we can’t use the familiar approach and/or deploy common strategies. We need to step up our game.
1. Data Silos and Integration
Start-ups and scale-ups often adopt multiple tools and platforms quickly, leading to fragmented data spread across various systems (CRM, ERP, marketing tools, etc.). Integrating this data into a cohesive system is complex and resource-intensive. This is especially true if you fail to a) invest in data integration platforms, and/or b) develop a unified data architecture early on.
In all honesty, a tech leader’s hands are often tied either due to budgetary restraints or late arrival. Consequently, disconnected data sources hinder holistic insights and create inefficiencies in decision-making and you can’t exactly “correct” what’s been done wrong right from the start on short notice.
How to solve this problem?
When traditional mitigation strategies are not viable, you can still take alternative, resource-efficient steps. These approaches focus on leveraging existing resources, prioritising immediate needs and adopting creative low-cost solutions.
1.1. Manual Integration with Pragmatic Prioritisation
Identify the most critical data silos that impact decision-making and prioritise integrating those first. Use lightweight manual processes or scripting (eg, Python, Google Sheets) to consolidate data where automation tools are unavailable.
From that point onward, do the following:
Conduct a quick audit to map critical data flows and prioritise based on business impact.
Use basic automation tools like Zapier, Make (formerly Integromat) or built-in export/import features of existing platforms.
Focus on incremental improvements—address key bottlenecks rather than aiming for perfection.
The outcome of these measures should be partial but impactful data integration for essential use cases without significant resource investments.
1.2. Leverage Existing Tools and Free/Open-Source Options
Maximise the utility of existing platforms and adopt free or open-source tools for basic data integration. Your sequence of actions should be like this:
Explore native integrations provided by current software (eg, APIs, built-in connectors).
Encourage teams to utilise data exports, shared dashboards or reports from existing tools.
This should result in cost-effective integration with tools already in your tech stack.
1.3. Empower “Data Stewards” Within Teams
If you are in a larger organisation, identify key individuals within departments who can take ownership of their team’s data. These people should act as intermediaries to share and consolidate information.
Now, to make this process as smooth as possible, take the following steps:
Designate a “data steward” in each team to document, clean and standardise departmental data.
Create simple workflows or templates for data-sharing (eg, shared Excel sheets or cloud folders).
Facilitate regular meetings where data stewards align on metrics and share insights.
What you are looking to achieve with this is not only improved communication but also understanding of data across departments without requiring centralised systems. It is a longer walk around, no doubt, but on the bright side, it will create a data processing singularity in the long run.
1.4. Adopt a “Federated Data Governance” Model
At first glance, this solution seems like it might lead to a pinball effect, with you bouncing from one office to another in a desperate search for that final document. Be that as it may, if you allow teams to maintain control over their own data while introducing light governance structures, it will a) reduce silos, and b) result in shared standards and definitions. However, it won’t happen on its own so to achieve those results, follow this strategy:
Define a small set of core metrics or KPIs that all teams must report consistently.
Provide teams with guidelines for data structure, format and reporting (eg, a standard CSV template).
Finally, use simple collaboration tools (eg, Slack, Notion) for sharing updates and insights.
And there you have it – a fully decentralised yet coordinated approach to data management that minimises silos. Because sometimes, even the government’s bureaucracy turns out efficient.
1.5. Pilot Low-Cost Data Lake
If — and this is a big if — resources allow for at least minimal investment, pilot a low-cost, pay-as-you-go cloud data lake solution. You want a focused, incremental approach to centralisation without incurring large up-front costs.
Gradually migrate the most critical data into the data lake while leaving less critical silos untouched.
Later, during a fast-growth stage, when you get your hands on more resources, this can easily evolve into a full-stack cloud data storage and processing.
1.6. Create a Cross-Functional Data Task Force
As you can assume, this strategy perhaps better fits the onset of the fast-growth stage, but it could also be just what you need in your start-up. This is how it works:
First, you start by forming a small task force with representatives from key teams to collaborate on solving integration challenges (not a full data team).
Then, you task the team with regularly consolidating reports or insights and aligning metrics.
Finally, they share consolidated data via basic tools (eg, Google Drive, Notion, shared dashboards).
It is an agile team effort that minimises dependencies on expensive tools or specialists.
The core philosophy here is: start small, build incrementally.
In other words, when constrained by budget or timing, focus on solving the highest-impact problems first. Admit to yourself that perfect integration may not be possible, but incremental improvements can still provide meaningful value. By being a bit creative and by maximising existing resources, technology leaders can mitigate the impact of silos without requiring substantial investments.
2. Data Quality and Accuracy
Your most immediate challenge is the all too familiar consequence of rapid growth and that’s a lack of consistent data governance. As you know, this inevitably leads to poor data quality (inaccuracies, duplicates or incomplete data).
The impact can turn out devastating because low-quality data undermines the reliability of insights, leading to poor strategic decisions. Imagine a marketing team missing an entire segment of the target audience or misaligning the core message. Sooner than later, all fingers will point at you.
On a normal day, you would mitigate by:
Implementing data validation and cleansing processes.
Establishing data governance frameworks.
Regularly auditing and updating data sets to ensure accuracy.
But remember, this is not your normal day. More often than not, technology leaders inherit a chaotic environment with poor processes and must react instead of being proactive.
Here’s what you can do in such a situation:
2.1. Triage the Data Chaos
Your immediate priority is to identify the most critical areas where poor data quality immediately impacts the company’s survival. Take the following steps:
Conduct a rapid audit of key data pipelines and processes.
Focus on revenue-critical systems (eg, billing, sales forecasting, customer data).
Prioritise data that directly affect regulatory compliance, financial reporting or mission-critical KPIs.
In the end, you will understand where to focus efforts for maximum impact in the shortest time.
2.2. Deliver a Few Quick Wins to Build Credibility
In other words, identify and solve one or two highly visible data issues to demonstrate progress and build trust. Simply fix a problem that has frustrated key stakeholders (eg, cleaning up sales pipeline data or resolving overdue billing errors) and then publicise the success with tangible results (eg, “Resolved 300 duplicate records, improving invoice accuracy by 20%”).
And now you have improved stakeholder confidence and momentum for broader changes.
2.3. Implement a “Minimum Viable Governance”
Quickly enforce lightweight rules to address the most damaging data quality issues without overengineering. This is achieved by:
Defining non-negotiable standards for critical data fields (eg, customer IDs, transaction amounts, dates).
Creating simple validation scripts to flag obvious errors (eg, missing fields, incorrect formats).
Using tools already in place (eg, Excel, SQL, lightweight automation tools like Zapier) for basic cleaning and validation.
If you do everything right, you should end up with an immediate reduction in errors, enabling more reliable decision-making.
2.4. Mobilise a Data “SWAT Team”
This strategy is more appropriate for larger organisations, but it can be scaled down to fit the purpose of a start-up.
In essence, you assemble a cross-functional, small team with representatives from critical departments to act as a task force. To succeed, this is what you should do:
Identify power users or, as some call them, “data champions”, from key teams like finance, operations and marketing.
Assign clear roles: one focuses on cleaning sales data, another on financials, etc.
Empower them to fix data in real-time and escalate issues to you directly.
The outcome is rapid, team-based problem-solving that restores operational functionality.
2.5. Apply a “Spot-Fix and Lock” Strategy
In other words, fix the most critical data issues in high-priority areas and immediately lock processes to prevent further degradation.
Start by identifying high-impact errors (eg, duplicates in customer records, incorrect pricing). Once you identified the set(s), correct these errors manually or via scripts. Finally, implement basic process locks, such as requiring specific fields to be filled before records are saved or restricting edits to validated data.
You end up with stabilised data quality in key areas, reducing downstream chaos.
Once the immediate chaos is controlled, start laying the groundwork for systematic improvements and building a foundation for sustainable data management. For instance, create a roadmap for addressing root causes (eg, better governance, new necessary tools). But whatever you do, don’t forget to document lessons learned from the crisis to guide future processes.
The key principle here is: stabilise, not perfect.
Remember, your goal is to bring enough order to stabilise operations and decision-making, even by using imperfect solutions. Once the immediate crisis is averted, you can gradually transition to proactive long-term strategies.
3. Scalability of Data Infrastructure
Let’s see what we can do with infrastructure bottlenecks caused by over-relying on basic tools that now can’t handle the exponential growth of data as the organisation scales. Instead of smooth operations, we have slow analytics processes, delayed insights and increased costs because systems struggle to keep up.
Again, on a normal day, you would simply:
Adopt cloud-based, scalable data storage and processing solutions.
Use modular systems that can grow with the organisation.
Plan for scalability when designing data architectures.
But that simply isn’t the case. Your predecessors (if any), didn’t quite do the job right and now you have a serious problem – unscalable data in a fast-growing company.
When faced with such an infrastructure in a rapidly growing organisation without the resources to invest in modern solutions, you must focus on triage, optimisation and tactical solutions. The goal is to stabilise the infrastructure to support growth in the short term while preparing for future scalability once resources are available.
3.1. Triage the Infrastructure Bottlenecks
Your priority is identifying the most critical bottlenecks in the current infrastructure that directly impact operations or decision-making. That is, perform a rapid audit of the existing infrastructure to identify pain points (eg, slow query response times, system outages, capacity issues).
Once identified, prioritise fixing the systems that handle mission-critical data (eg, sales, billing, customer support).
This should give you a clearer understanding of where to focus limited resources for maximum impact.
3.2. Optimise Existing Resources
While you are already dealing with bottlenecks, activate the afterburner by squeezing the maximum performance out of the existing infrastructure with targeted optimisations.
For example:
Database Tuning:
Optimise query performance by indexing critical columns, rewriting inefficient queries and archiving old data.
Partition large tables if possible to improve performance.
Storage Management:
Compress data to reduce storage requirements.
Move cold or historical data to cheaper, offline storage (eg, local hard drives or NAS).
Batch Processing:
Shift non-urgent data processing tasks (eg, report generation) to off-peak hours.
If done correctly, you should see immediate performance improvements without requiring new infrastructure.
3.3. Implement Stopgap Solutions
The play here is to introduce temporary fixes to alleviate pressure while preparing for longer-term improvements.
Here’s what you can do to achieve this:
Use local servers or existing hardware more efficiently (eg, repurpose underutilised machines as temporary data nodes).
Set up lightweight, open-source tools for specific needs (eg, Apache Kafka for message queuing, PostgreSQL for database expansion).
Leverage basic automation tools to reduce manual intervention in data handling.
These solutions may appear trivial but keep in mind what we are trying to achieve here and under which circumstances. We ultimately want stabilised infrastructure to support ongoing growth, even if suboptimal.
3.4. Segment and Prioritise Data Loads
Data don’t need to be processed or stored at the same priority level. Therefore, segregate data workloads based on their importance and urgency. For example:
Categorise data into tiers (critical, operational, historical).
Allocate the best resources to the most critical data sets.
Limit real-time processing to essential data and defer non-critical processing.
The cumulative effect is reduced strain on the infrastructure without sacrificing business-critical operations.
3.5. Leverage Community and Open-Source Resources
Sometimes, you don’t have any other choice but to enter the dark ally of open-source tools and use them to address specific pain points in the data infrastructure.
Use open-source tools like MySQL, PostgreSQL or SQLite for additional database capacity and implement lightweight ETL solutions like Apache NiFi or Singer for data integration. Finally, make sure to monitor system health with, for example, Zabbix or Prometheus.
None of us prefer open-source solutions, but they are cost-effective and scalable enhancements. For instance, we are utilising Mautic as our central nervous system and a single source of truth. Our CTO, Jason Noble, spent a lot of sleepless nights getting that open-source beast to life and keeping it updated. However, it was worth it. We don’t spend thousands on monthly subscriptions and we alone own all data. Would it be the same if we had chosen HubSpot, for example, that’s highly questionable.
3.6. Build Manual Processes as Interim Solutions
When automation or scaling proves impractical for any number of reasons, use manual processes to handle critical data workflows.
You simply assign dedicated teams or individuals to manage data flows that the current infrastructure cannot handle (eg, manually consolidating reports or transferring data between systems). Just remember to use templates or scripts to streamline repetitive tasks.
It’s not exactly practical and can cause delays, but these short-term solutions keep the business running without overwhelming the infrastructure.
The key principle here is: survival first, perfection later.
In this critical phase, focus on stabilising the infrastructure and ensuring business continuity. While the current environment may remain suboptimal, these actions will buy you time to secure the resources and strategic alignment necessary for sustainable, long-term growth.
And remember, no matter the situation, begin laying the groundwork for scalable solutions even if resources are tight. Begin consolidating fragmented systems into a single source of truth wherever feasible. Also, document the current infrastructure and create a lightweight plan for migration to a scalable architecture once resources become available. And in that little spare time you get around lunch, try to identify low-cost, incremental investments that could ease scalability bottlenecks.
4. Talent Shortages and Skill Gaps
Start-ups often struggle to attract and retain skilled data professionals due to competition from larger organisations. That lack of expertise can result in underutilised data assets and suboptimal decision-making.
Commonly, a CTO would deploy these three strategies:
Upskilling existing team members in data literacy and analytics.
Partnering with external consultants or leveraging outsourcing for specialised needs.
Cultivating an attractive work culture to retain data talent.
Now imagine the scenario in which none of the proposed mitigation strategies works, at least not in the long run because the small team of only a few simply can’t find additional time to upskill in data literacy and analytics (they are software engineers). Partnering with external consultants or some extensive outsourcing is out of the question and the work atmosphere is so grim that it is impossible to create and cultivate an attractive work culture to retain data talent. But the paycheck on the other hand is so big that you don’t want to quit and search for something else. What can you do?
Here is the list of the most realistic strategies:
Identify the smallest set of tasks that deliver the most significant results and focus only on those.
Empower non-technical staff to handle basic data-related tasks with user-friendly tools.
Accept that the data infrastructure and processes won’t be perfect and focus on “good enough” solutions.
Create opportunities for your team to learn informally and in small increments, without requiring extensive upskilling efforts.
Collaborate with other departments to share responsibilities or gain access to additional skills.
Improve communication about current constraints and challenges to align expectations.
If possible, bring in limited short-term help from freelancers or contractors for specific tasks.
Implement changes that yield long-term benefits without requiring ongoing maintenance.
Even in a grim atmosphere, recognise and reward your team’s efforts to boost morale.
As you can see, the guiding principle here is: stabilise to survive. In other words, if you are in a highly stressful and negative environment with limited resources and a small overburdened team, just focus on stabilising the situation and delivering “good enough” results.
Therefore, prioritise ruthlessly, automate strategically and leverage creatively to ensure the team survives the current challenges while laying the groundwork for future improvements.
5. Balancing Speed with Rigour
As we said early on, start-ups and fast-growing organisations are often forced to make quick decisions to maintain their competitive edge. This leads to shortcuts in data analysis or overreliance on intuition.
Normally, a technology leader would implement these three strategies to balance speed with rigour:
Create streamlined yet robust processes for data validation and analysis.
Foster a balance between agility and thoroughness in decision-making.
Encourage cross-functional collaboration to validate insights before acting.
But what happens when data silos hinder speed and rigour while pressure for speed amplifies silos?
Let’s use case studies to better understand this causal relationship:
Scenario 1: A start-up rushes to launch a new product. Sales and marketing teams use different platforms to track leads and engagement. Decisions about the product’s target audience are made based on siloed data, leading to misaligned messaging and wasted resources.
Scenario 2: A scale-up prioritises speed in reporting but lacks a unified data warehouse. Analysts spend time manually consolidating data, delaying insights and increasing the risk of errors, which undermines rigour.
How to break this vicious cycle?
In ideal circumstances, organisations would employ the following strategies:
Adopt centralised data platforms or warehouses early on to enable seamless access across teams.
Encourage teams to adopt scalable systems even if they take longer to implement initially.
Establish cross-functional practices by facilitating data sharing and strategic alignment between teams.
Only, we are not that lucky. There are no warehouses, teams still work on legacy (read: rigid and fixed-capacity) systems and nobody shares anything. It even seems that teams pursue different strategic goals. That’s the situation we met after accepting the role.
What we need now is a phased, tactical approach that delivers quick wins while laying the groundwork for broader transformation. It is essentially a five-step strategy:
Step 1: Triage and Stabilisation
In this step, our priority is to identify critical interdependencies so we can get some clarity on immediate priorities to stabilise the situation.
To find out, we can conduct a rapid assessment of the most critical pain points. For example:
Which decisions are being delayed or compromised due to silos?
What strategic misalignments are most damaging to the company?
Then, we need to focus on cross-functional bottlenecks where silos directly affect speed and rigour. This requires the creation of a temporary “Data Task Force” or a small agile cross-functional group that will address critical silos by accessing and consolidating data needed for immediate priorities. The good practice here is to assign members from key teams (eg, product, finance, operations) to represent diverse perspectives.
Eventually, all these efforts should create a temporary workaround that will enable collaboration and quick fixes.
Step 2: Quick Wins to Build Momentum
Start by creating a “Minimum Viable Integration” to achieve basic data sharing without major resource investments. That is, use lightweight solutions to connect siloed systems, focus on critical data flows and automate repetitive processes.
Next, establish a “Single Source of Truth” for critical metrics to enable shared visibility into business performance, fostering alignment.
Finally, pilot cross-functional decision reviews for high-stakes decisions to create a foundation for a gradual cultural shift toward collaboration and shared accountability.
Step 3: Establishing a Foundation for Change
To reduce strategic misalignment and increase clarity, teams must unify under the same goal framework. To get there, team leads need to be aligned on well-defined company-wide strategic goals. These goals must then be broken into measurable objectives tied to specific team deliverables.
It’s only now that you can start prioritising tactical investments in scalability by implementing high-impact, low-cost upgrades to legacy systems (eg, replacing outdated software with lightweight cloud-based tools).
You can easily justify these investments by linking them to business outcomes like faster time-to-market or improved customer satisfaction. Just remember to start small to fit within resource constraints.
The outcome is gradual modernisation without overwhelming the organisation.
Step 4: Cultural and Process Transformation
You want to achieve three goals here:
Incentivise data sharing to reduce resistance to collaboration and improve data flow.
Simplify and streamline processes to improve operational efficiency without introducing unnecessary complexity.
Drive a mindset shift (lead by example).
Step 5: Measure and Adjust
What to track and measure?
Well, track key indicators such as decision turnaround times, collaboration frequency and strategic goal alignment. Use these metrics to gauge the effectiveness of your interventions. Just remember to regularly share progress updates with leadership and the broader team.
How to adapt for scaling?
Build on early successes to expand collaboration and data-sharing practices.
Gradually phase out legacy systems, reinvesting savings into more scalable solutions.
Adjust priorities based on the evolving needs of the organisation.
The result is sustained momentum and long-term scalability.
Conclusion
In challenging environments, maintaining data integrity for strategic planning requires a balance between stabilising immediate risks and building a scalable foundation for the future. Quick wins, collaboration and adaptability are essential to breaking the cycle of dysfunction and driving sustained organisational success.
The key takeaways:
Understand and prioritise immediate risks.
Establish quick, practical solutions.
Promote collaboration and alignment.
Balance speed with rigour.
Leverage existing resources creatively.
Drive cultural transformation.
Measure progress and adapt.
Through four weeks and sixteen lectures in Module 8 of our Digital MBA for Technology Leaders, the faculty of senior executives responsible for data management in their organisations, teach this and other subjects in much more detail, using years-long experience. You will learn how to adjust to an array of different circumstances to, ultimately, maintain data integrity even in worst-case scenarios.
It’s challenging to create a truly immutable decision-making framework, especially in dynamic environments with conflicting priorities. However, you can create a robust and adaptable framework that provides consistent guidance while allowing for flexibility when needed.
Here’s a possible approach if you are managing two conflicting departments dependent on each other’s productivity (eg, CPTO role):
1. Establish Clear Objectives and Metrics
Define overarching goals that both departments contribute to. This fosters a sense of common purpose and encourages collaboration.
Establish clear, measurable metrics for each department that align with the shared goals. This ensures accountability and clarifies expectations.
Identify metrics that reflect the interdependency between the departments. This could be on-time delivery, project completion rate or shared resource utilisation.
2. Create a Decision-Making Process
Establish a regular meeting or communication channel where both departments can discuss issues, share updates and make joint decisions.
Encourage the use of data and objective analysis to inform decisions, reducing emotional bias and promoting fairness.
Define clear escalation paths for resolving disagreements, ensuring that conflicts are addressed promptly and effectively.
3. Foster a Culture of Collaboration
Align reward systems and incentives to promote collaborative behaviour and recognise joint achievements.
Encourage open and transparent communication between departments, fostering trust and understanding.
Provide training on conflict resolution techniques to equip employees with the skills to manage disagreements constructively.
4. Periodic Review and Adaptation
Conduct periodic reviews of the framework’s effectiveness, soliciting feedback from both departments.
Be prepared to adapt the framework as needed to accommodate changes in business objectives, organisational structure or external factors.
Practical Application: Engineering & Product Development
As a CPTO, you are leading both Engineering and Product Development teams. Product Development designs new features and products, while Engineering builds and implements them.
Shared Goal: Successfully launch innovative, high-quality products that meet market needs and achieve business objectives (eg, increased revenue, user growth).
Individual Metrics:
Product Development: Number of features designed, user stories defined, prototypes created.
Engineering: Velocity (features delivered per sprint), code quality, system uptime, bug resolution rate.
Interdependency Metric: Number of features successfully launched and deployed without major bugs or delays.
Decision-Making Process:
Weekly joint meetings to review product specifications, discuss technical feasibility, estimate development time and identify potential roadblocks.
Decisions are driven by data on development capacity, technical constraints, user feedback from previous releases and market research.
A clear escalation path is defined for resolving disagreements, involving a technical lead and a product manager.
Scenario:
Product Development proposes a complex new feature with a tight deadline. Engineering raises concerns about feasibility and potential impact on system stability.
Framework in Action: In the joint meeting, both teams present data: Product Development shows market demand and potential revenue impact, while Engineering presents data on current workload, technical challenges and estimated development time.
Outcome: Through collaborative discussion, they might decide to adjust the scope of the feature, extend the deadline or allocate additional resources to ensure successful implementation.
As you can see, this framework fosters a collaborative environment where Engineering and Product Development work together effectively to achieve shared goals. It encourages data-driven decision-making, clear communication and proactive problem-solving.
It is a kind of decision-making framework commonly utilised by Chief Product & Technology Officers but can be adjusted and applied to any intersection.
Rebecca Murphey, the Field CTO at Swarm, delved into improving developerproductivity in a recent CTO Shadowing session we hosted.
However, she did not focus on zooming in on individual developers even though there’s talk on LinkedIn about how some 10% of them aren’t doing any work. Instead, Rebecca dealt with teams working together to ship value to customers and, more importantly, how to further improve that process.
Because, at some point, you stop valuing people’s contributions in terms of time, and start valuing their contributions in terms of life goals. In other words, you start valuing the outcomes they produce that may expand beyond actual coding or otherwise expected output.
Additionally, Rebecca looked into the overall developer experience (not to be mistaken for ping-pong tables and kegs) and the concept of business outcomes in terms of its correlation to developer productivity.
The Guiding Principle for Improved Developer Productivity
The guiding principle is simple:
We want our developers to work efficiently on the right things.
For that to happen, all three concepts: productivity, experience and outcomes, must come together to lead to a successful engineering organisation. And it all depends on tech leaders and their ability to set the environment and processes right.
So for a starter, ensure that your team isn’t working on twenty things at once.
Second, if they are drowning in interruptions, they must have the capacity to reduce them.
If interruptions occur regularly, ask yourself the following questions:
Did the organisation set clear and reasonably consistent priorities?
Are the teams empowered to follow those priorities and seek the outcomes the business wants?
Are teams allowed to say no?
Where Leaders Often Go Wrong and Damage Productivity
Teams are forced to work on too many things at once
Missing automation processes
RULE OF THUMB: A team of four people should work on one story before moving to another instead of having each member work on a separate story.
Habits and Tendencies of Engineers – US vs. Outside the US
Cultural differences are profound when it comes to productivity. In the US, for instance, people tend to work more independently; i.e., less collaboratively. They want to own a project so that they can brag about it at their performance review in the hope of getting a bonus.
Outside of the US, on the other hand — and this is a vast generalisation –, there is a lot more collaboration. In other words, people tend to work together on something as a team.
The same goes for monetary incentives: they are more appreciated in the US than outside the US.
Preventing Common Failure Modes
As a leader, there’s a lot that you can do to intervene in some of the most common failure modes:
Prevent working on too many things.
Defend against interruptions.
Reduce “waste work” (the result of the constant switching or wasting time on deprecated projects).
Fail to address these issues and the outcomes are:
Less predictable delivery
Higher error rates
More wasted work
Flow Efficiency
Flow efficiency measures the proportion of active time spent on a task compared to the total time it takes to complete the task, from initiation to production.
Ideally, you want to see around 80% active and 20% idle time.
If, on the other hand, you get 30% active time and 70% idle time, either you are utilising about 30% of the capacity or you’re trying to do too many things. Also, the team might be too small to handle so many pull requests in such a complex system. Or, you have an optimal-size team, but there are too many external processes they have to go through (eg, security or API reviews, manual QA process…).
How to Define Capacity
Here, we are talking about predictability ratherthan productivity. It is a more suitable term because to determine the capacity, you must know the likelihood of having something finished in X days.
By default, this assessment is based on historical performance. For example, you know that your team can deliver five small stories a week. Hence, the predictability gives you their capacity.
TIP: Use Gen AI to build the model. Include variables such as flow efficiency, delivery and lead time and failure rate.
The other thing that helps with capacity planning is to ensure you’re never doing anything big. That, of course, doesn’t mean that you never accomplish anything big. Instead, you work on small things that lead to big things. Here’s an example:
You can ship one ticket at a time, but the feature may take 20 tickets before it’s done. Now, this number might seem insignificant; however, don’t forget that you’re a) constantly putting that code in production, and b) making sure that the right people can see the current state of it even though your customers perhaps can’t. When you can redo this process, it is called reduced batch size.
When you can reduce the batch size and get each of your units of work down to about the same size, your predictability goes up and your capacity becomes more predictable.
Let’s go even deeper now:
We know that we can get five stories done every week. Here’s the key: Every story should be 1 to 2 points, not 8 or 13. As soon as you start talking about eight and thirteen points, you have walked away from predictability. In other words, larger stories have a greater likelihood of being carried over to the next sprint.
Speaking of sprints…
Sprints are great training wheels, but once you get basic alignment and prioritisation in place, Kanban can be a lot more effective because it reduces the ceremonies and focuses the team on a specific goal. However, bear in mind that Kanban might be more suitable for mature teams because it focuses on predictability.
TIP: If you’re using Jira, you can make the column red when you have too many things in progress and/or review.
Now we are getting to the part where you acquire the relevant data.
The “Brains Methodology†Framework
The principle of the Brains Methodology is to get a baseline using DORA metrics for different teams (the four metrics that look at the tension between delivery velocity and delivery quality) and then assess the current state.
Now, arguably the best approach here is to just sit and watch your team(s) working; especially if you have a situation where your lead time is one hour or something like that, but every deployment is broken. You want to understand why it is happening.
You’d be surprised how much you learn about your teams and processes if you just sit and watch.
Now, here’s the thing. You want to speak to each member besides doing surveys because you want to understand individual pain points and get their opinions on improvements. For example, you want to know what frustrates each of them daily while they work on projects and/or systems.
Once you have the answers, use them to implement changes and show the team that their feedback leads to concrete actions.
For example, you might want to increase productivity by 20%. One way to achieve that is to actively protect 40% of their time by acting as a shield against, say, product or sales teams that are trying to infringe on that time. This shielding helps them with problem-solving because they know that they can put their brains together without being interrupted.
On the technical side of things, you could implement a CI/CD process, get automated tests up and running and place alerts in your production.
However, the most positive changes come from cultural shifts.
Take Stripe for an example. The company invested in developer productivity early on thanks to one engineer who realised that he could be more productive working on the builds than on the features using those builds. That gave birth to the platform and had an enormous impact on how work gets done at Stripe even today. The moral of the story is to let them choose their battles from time to time.
Remember, it’s better to focus on outcomes than on outputs; for example, an effective delivery that lifted net retention by 3% or reduced the cost of customer acquisition by 1%.
How to Subtly Introduce Brains Methodology?
(Without making everyone feel watched and having their every commit scrutinised.)
Start by explaining that this is about transparency and not things being put under the microscope. And that transparency will eventually provide evidence for their claims.
For example, an engineer might be frustrated with constantly working on keeping the lights on (KTLO). This type of work often limits opportunities to take on projects that would lead to a good performance rating. The team might also be concerned about their career prospects if they spend up to 80% of their time KTLO-ing. The Brains Methodology can help alleviate these concerns by providing clear evidence of how their time is actually being spent. This data can then be used to justify requests for additional resources or to support arguments for shifting priorities towards more innovative projects.
Of course, every now and then, you have to remind them that we live and operate in capitalism and that they are paid for the value they bring to the organisation. But when it gets hard for them, they must feel free to inform you about their predicaments because you’d want to know about it so that you could figure out what needs to change.
Now, as you can imagine, this transparency will shine a light on the imposed processes more than it will on teams. And that’s a good thing because more often than not, it’s the process that impedes the progress. Brains Methodology gives you a developer productivity tool to not only see inside a single process but also to get an overview of the intricate correlations and dependencies between multiple processes at once.
That’s why we are looking into team performance rather than individual. And that’s why we should be more interested in outcomes than outputs. Something to contemplate, isn’t it?
This article is based on a CTO Shadowing session with Bryan Seely, an ethical hacker and cybersecurity expert. Bryan is a former marine who, by his own admission, wiretapped the US Secret Service and FBI. Later, he worked with John McAfee and Mark Cuban and founded the Black Hat Conference in Riyadh in 2021.
Importance of Personal Hygiene in Cybersecurity
According to Bryan, there is a measurable and quantifiable number of ransomware strains that check for the Russian language as a second or a first language on your keyboard. So if you have a Russian language set as a first or second language, they won’t infect your machine.
Installing Wireshark should have the same effect because they’ll think you’re a honeypot because hackers don’t want you to figure out how they are doing things.
This just goes to show how important it is for technology leaders to closely follow cybersecurity news and updates.
Tips for Technology Leaders and SysAdmins
Password length must be over 14 characters.
Encourage security fundamentals, but don’t force it. Instead, do it incrementally because people tend to resist the sudden change. As a rule of thumb, never change more than 10% of the framework in a single attempt and people will think they are part of the solution and the team that is planning everything. This approach will also prevent overload on the team implementing migration.
When evaluating a new technology, make sure it does not contain too many CVEs right off the bat. For example, a biometric fingerprint scanner without supervision.
Stay informed about the latest threats and security news (during the session, Bryan suggested Krebs on Security blog).
Biometrics work, but 2FA must be mandatory. Almost every single big breach was enabled by negligence (eg, leaving credentials to a VPN open for anyone to see them).
Shut down access immediately upon exit or predefined (read: relatively short) idle time. You can easily find yourself in a situation where you don’t have the slightest idea about an entry point which will leave attack vectors open simply because someone forgot to shut something down or close the ticket.
Never use built-in password managers.
Don’t trust an app’s permissions requests; in most instances, your consent is irrelevant and the app will pass the information anyway.
When training employees, always use live training instead of videos.
Cybersecurity Challenges in Quantum Computing
According to Bryan, there is a great chance of someone breaking encryption under anyone’s radar. In other words, no one will be aware of the exploit.
Many who are counting on the advanced analytical and detection capabilities of an AI should realise that they don’t actually have the AI but merely a bunch of what-if statements nested in 19,000 lines of code. — Bryan Seely
Conclusion
Cybersecurity is not just about technology, but also about vigilance and informed practices. Proactive steps and continuous learning are your best defence in the ever-evolving cybersecurity landscape.
