Designing a data platform for a new product or service is a fascinating and exciting challenge. It’s also a daunting one. Instead of plugging a new feature into an existing platform that is proven and reliable, the onus is on you. You have to start from scratch and create a platform that becomes just as proven and reliable.
Like I said: daunting.
From our wide experience of helping clients in the retail, financial services and gaming sectors, here are the top ten factors we always consider.
1. New technology
This doesn’t mean implementing every possible new feature or design methodology. Instead, it’s about gaining the knowledge required about the latest technologies available so that the most appropriate decisions are taken when building the foundation of a solution. Existing functionality may already be offered as part of a third party system or service, for example, which may eliminate the need for development while providing a cost-effective, managed, secure and scalable option.
2. High cohesion and low coupling
A design in which functionally can be separated with relative ease into components performing specific tasks will enable a far greater flexibly. The ability to replace or adapt parts of an application will allow for quicker re-architecture and for third party solutions to be incorporated.
The ability to test in an automated and repeatable manner the full scope of the design is absolutely critical, as without it there is an increase in both unknowns and risk.
The limitations of physical resource will affect the design more than any other factor. Inconsistent network communication or high latency links, for example, will make architectures that perform high volume or synchronous transactions impossible to support.
As design complexity increases, there is often a point where it becomes subject to financial restrictions that make further development or expansion no longer viable.
It’s important to understand the technology being implemented and how it can be used. A design that pushes the limits of a technology can work in many cases, but thought must be given to how far a design is pushed and the restrictions that are present. For example, real time synchronization across a global infrastructure may quickly hit limitations.
A design may achieve many of the core requirements of a business in redundancy and high availability by an infrastructure which is globally dispersed, but this may be impracticable because of legal requirements to have data hosted in geographical locations.
Deciding how a design will scale is almost as important in many ways as the functional ability of the solution. A solution designed with little thought for scalability usually relies on vertical scale-up strategy that can very quickly hit both resource and financial limitations. A horizontal scale-out solution, however, which incorporates sharding or data partitioning will scale to far higher levels as the shared load is easier to scale.
A system that may seem incredibly secure today will be obsolete tomorrow as technology and research is undertaken. It’s important to make sure that security is implemented at a level that meets the requirements, but it’s prudent to exceed those requirements up until the point where it becomes impossible in terms of resource. A secure system is always most vulnerable at the weakest point and implementing minimal security only guarantees that.
Plan how to design the platform using an iterative approach to ensure goals are achievable and well thought out. A plan should be based on what needs to be achieved and not what time allows. If a plan is shaped by additional factors, these should be highlighted and risks identified.
|Coeo’s platform consultants can help you define and design
a data platform that is operationally right for your business.
Find out more about us.
This is a guest post from Coeo. Europe's most trusted analytics and data management expert, Coeo is the number one provider of database strategy in the Retail, Financial services and Gaming industries, and delivers technology strategy and support for businesses who need to get the most from their data.
The Coeo team hold more Microsoft certifications than any other data platform specialist in Europe and are passionate about sharing their knowledge and expertise to help customers become industry leaders.
Also in Blog
Database DevOps has come of age. Now seen as a key technical practice which can contribute to the successful implementation of DevOps, it stops the database being a bottleneck and makes releases faste...
Also in Database development
DBAs have a tough job sometimes. When deployments go wrong, they’re the ones everyone looks to. They’re also the ones who have to fix it, which can mean working through the night, the weekend, t...
Also about data platform
In the last few years, Black Friday and Cyber Monday have become synonymous with both getting a good bargain and causing retail pandemonium. However, times are changing. News items about people queuei...