I have been charged with building back end reporting systems to reconcile our data with the partner. I took the day off today so had time to reflect on this. It's a thought that surrounds my head each day like a malaise that I can never fully pinpoint or define. Until now. I assume by human nature that my experience is similar to any other fast growing companies where it is more vital to get product out there than to cover all the use cases and meet all the requirements/needs of all the stakeholders. When you run a company lean (lots of work not a lot of resources), you can miss some damn important things. Not important in the heat of battle but important if you don't go back and address the issues and fix root causes as your company grows. These things can actually throttle your ability to grow. But it was more important at the time that you grew - get big fast - and the assumption is that you can fix these problems later. One day, the more common issues will be fixed up front when companies prove that they can get big faster than other companies who subscribe to the old philosophy... Back to dead bodies... I have found through data analysis that in most cases, I can not build a back end reconciliation system against existing transactional systems. I will more often than not find a major hole where details about a transaction aren't recorded. Sometimes, if we are lucky, someone else recognizes that the transaction processing system is not capturing certain information and tries to capture this information themselves, manually, in a spreadsheet. Problem is at this point, the information collection is prone to human error and inconsistent data standards. This is where I come in. I have to survey all available data and build a reconciliation system to show where all the money went and in the case of some companies settle up with which partner owes which partner money. A word to the wise, if your reconciliation system involves sourcing data from a manually maintained spreadsheet, you've got major problems. So now, my job is not just building reporting systems but also finding dead bodies. These dead bodies are flaws in the transactional system design that prevent us from reconciling properly with some partner. If the company culture is good, it gives you a forum for identifying root cause issues and prioritizes the fix of root cause issues over the priority to build a reporting system against a flawed transactional system. In the real world, there are other forces at work, regulations, politics, etc. Regulators will say - you should have been doing this the right way in the first place.Partners will too. They'll ask, why can't you figure out how to square up the money you owe me? Well, you can't say my transaction system is flawed because I didn't spend enough time on it because I was busy building all the other features you were asking me to build. So what you have is dead bodies and someone needs to clean them up. In my experience, I have been able to identify issues in transactional systems and build band-aids to collect data properly and then let managers know that a more robust system needs to replace my makeshift system. Lots of effort goes into fixing something designed incorrectly in the first place.