FEATURE : SOFTWARE DEVELOPMENT
and will help manage the expectations of stakeholders . Managing expectations can make or break a project . This is one of the biggest lessons I have learnt . Being upfront with your stakeholders builds personal trust and manages expectations .
It is important to set boundaries and lay out terms of engagement . This protocol will prevent you and your team from being overwhelmed by ad-hoc requests and will help prioritise requirements and manage client expectations .
IN TRYING TO MAKE THE DATA AVAILABLE TO CONSISTENT FORMATS DESPITE THE ENDLESS VARIATIONS AND PROBLEMS , WE COULD NOT TAKE THE TIME TO UNDERSTAND THE CONTEXT OF THE DATA .
Technology can be intimidating , especially to nontechnical people . For any developers reading this article , consider the first time you come across a new technology stack or tool . That feeling of intimidation or imposter syndrome might be ever-present for someone who is non-technical . For your solution to be used , it needs to be simple with relevant automations .
In one of my projects , we built a JSON configuration layer to deploy data pipelines and their associated infrastructure with Terraform , one of my favourite tools . For us , it was relatively simple as all we had to do was populate the JSON config file , follow the Git workflow , Gitflow that we had defined , and everything would be deployed for us via CI , CD . Surely our users would easily adopt this pattern ?
People struggled with this . As a platform team , we made assumptions about the capabilities of our users and did not engage with them to understand where they were . Many people were unfamiliar with git which complicated many of our deployments . Reflecting on this , we probably should have created a simple user interface for our users with helpful prompts and automations to fulfil the required Gitflow .
In another project , an on-premises feature store was migrated to AWS . As Spark fans , we opted to rebuild the SQL scripts in PySpark with AWS Glue . We assumed that since Python is extremely popular , the barrier to entry was low for users to build their own custom scripts .
The reality is that you do not necessarily know the context of the people you are talking to . Job titles can be misleading and force assumptions into interactions . It helps to first understand if the person you are speaking to is technical or not . Secondly , what level of information do they need to know given the situation .
I made the mistake of speaking to a business stakeholder at my technical level . This led to miscommunication where we would end up talking past each other . After a month or so of this back and forth where we did not make much progress , and with some guidance from a colleague , I pivoted my approach and language to meet the stakeholders needs .
This streamlined our interactions and made for a far more fruitful and enjoyable project . This lesson is something I have applied ever since learning it , and while I sometimes find myself reverting to more technical language , I do my best to check in with the other person to ensure that they are following what I am saying .
Many people were unfamiliar with Python and Spark , and we should have made their lives easier by sticking to SQL . Perhaps we could have opted for a tool like DBT to keep this as simple as possible , with appropriate governance and data discovery to prevent DBT from spiralling out of control .
For proper adoption of your solution , you need to be guided by your user . This can be guided by a persona , the actual person who most accurately represents your userbase and whose opinions should guide your development . For example , perhaps they prefer a blue button over a green one and therefore , we should favour their feedback . One needs to build for what our users need versus what we think they need . Beyond this , with regular feedback you are more likely to hit your targets .
Use simple language and sentences , especially in documentation . Complex language will prevent people from understanding what you have built and may prevent it from being used . Images and flowcharts will often explain more than words .
36 INTELLIGENTCIO AFRICA www . intelligentcio . com