As we transition from one year to the next, it’s a season of reflection and looking forward. As an analyst, the end of the year is a time to learn from past work, analyze its outcomes and consider its potential impact on the future.

In 2023, enterprise data management (EDT) solutions underwent significant changes due to the influx of generative AI technologies. These technologies have fundamentally altered how businesses approach data management, analysis and usage. In this post, I’ll review some of 2023’s highlights in this field.

How Different Areas Of EDT Are Evolving

Over the past year, there have been promising developments in EDT across several key areas. These include data management itself, where the focus has been on using AI to improve how data is organized and accessed. The data cloud sector has also experienced growth, with more businesses adopting cloud-based solutions because of their flexibility, scalability and facility for integrating tools that handle unstructured data.

In data protection and governance, there has been a continuous effort to enhance security measures to safeguard sensitive information. Database technologies have also improved, particularly in handling and processing large data volumes more efficiently by incorporating generative AI.

Recent advancements in data integration and intelligent platforms have been geared towards better aggregating data from multiple sources, allowing for more comprehensive data analysis. The integration of AI and ML has further enhanced the capabilities of these platforms, improving data analysis interpretation and offering more profound and insightful analytical outcomes.

Full disclosure: Amazon Web Services, Cisco Systems, Cloudera, Cohesity, Commvault, Google Cloud, IBM, LogicMonitor, Microsoft, MongoDB, Oracle, Rubrik, Salesforce, Software AG, Splunk, and Veeam are clients of Moor Insights & Strategy, but this article reflects my independent viewpoint, and no one at any client company has been given editorial input on this piece.

Bringing AI To Data Management—And Vice Versa

“In a way, this AI revolution is actually a data revolution,” Salesforce cofounder and CTO Parker Harris said during his part of this year’s Dreamforce keynote, “because the AI revolution wouldn’t exist without the power of all that data.” Harris’s statement emphasizes the vital role of data in businesses and points to the increasing necessity for effective data management strategies in 2024.

As data becomes more central, the demand for scalable and secure EDT solutions is rising. My recent series of articles focusing on EDT began with an introductory piece outlining its fundamental aspects and implications for business operations. This was followed by a more in-depth exploration of EDT, particularly highlighting how it can benefit businesses in data utilization. These articles elaborated on the practical uses and benefits of EDT and its importance in guiding the strategies and operations of modern businesses.

As businesses continue to leverage generative AI for deeper insights, the greater accessibility of data is set to revolutionize how they manage information. This development means enterprises can now utilize data that was previously inaccessible—a move that highlights the importance of data integration for both business operations and strategic decision-making. For instance, untapped social media data could offer valuable customer sentiment insights, while neglected sensor data from manufacturing processes might reveal efficiency improvements. In both cases, not using this data equates to a missed opportunity to use an asset, similar to unsold inventory that takes up space and resources without providing any return.

Revolutionizing Data Cloud Platforms

Incorporating AI into data cloud platforms has revolutionized processing and analyzing data. These AI models can handle vast datasets more efficiently, extracting previously unattainable insights due to the limitations of traditional data analysis methods.

Over the year, my own collaborations with multiple companies suggested the range of technological progressions. As I highlighted in a few of my articles, Google notably improved its data cloud platform and focused on generative AI with projects including Gemini, Duet AI and Vertex AI, reflecting its solid commitment to AI innovation. Salesforce introduced the Einstein 1 Platform and later expanded its offerings with the Data Cloud Vector Database, providing users with access to their unstructured enterprise data, thus broadening the scope of their data intelligence. IBM also launched watsonx, a platform dedicated to AI development and data management. These moves from major tech firms reflect a trend towards advanced AI applications and more sophisticated data management solutions.

At the AWS re:Invent conference, I observed several notable launches. Amazon Q is a new AI assistant designed for business customization. Amazon DataZone was enhanced with AI features to improve the handling of organizational data. The AWS Supply Chain service received updates to help with forecasting, inventory management and supplier communications. Amazon Bedrock, released earlier in the year, now includes access to advanced AI models from leading AI companies. A new storage class, Amazon S3 Express One Zone, was introduced for rapid data access needs. Additionally, Amazon Redshift received upgrades to improve query performance. These developments reflect AWS’s focus on integrating AI and optimizing data management and storage capabilities.

Recent articles have highlighted Microsoft’s role in the AI renaissance, one focusing on the launch of Copilot as covered by my colleagues at Moor Insights & Strategy, and another analyzing the competitive dynamics in the AI industry. Additionally, Microsoft has expanded its data platform capabilities by integrating AI into Fabric, a comprehensive analytics solution. This suite includes a range of services including a data lake, data engineering and data integration, all conveniently centralized in one location. In collaboration, Oracle and Microsoft have partnered to make Oracle Database available on the Azure platform, showcasing a strategic move in cloud computing and database management.

Automating Data Protection And Governance

With the growing importance of data privacy and security, AI increasingly enables the automation of data governance, compliance and cybersecurity processes, reducing the need for manual oversight and intervention. This trend comes in response to the rise in incidents of data breaches and cyberattacks. AI-driven systems have become more proficient at monitoring data usage, ensuring adherence to legal standards and identifying potential security or compliance issues. This makes them a better option than traditional manual approaches for ensuring data safety and compliance.

Security is not only about protecting data but also about ensuring it can recover quickly from any disruptions, a quality known as data resilience. This resilience has become a key part of security strategies for forward-thinking businesses. Veeam emphasized “Radical Resilience” when it rolled out a new data protection initiative focused on better products, improved service and testing, continuous releases and greater accountability. Meanwhile, Rubrik introduced its security cloud, which focuses on data protection, threat analytics, security posture and cyber recovery. Cohesity, which specializes in AI-powered data security and management, is now offering features such as immutable backup snapshots and AI-driven threat detection; in 2023, it also unveiled a top-flight CEO advisory council to influence strategic decisions. Commvault has incorporated AI into its services, offering a new product that combines its SaaS and software data protection into one platform.

LogicMonitor upgraded its platform for monitoring and observability to include support for hybrid IT infrastructures. This enhancement allows for better monitoring across an organization’s diverse IT environments. Additionally, Cisco has announced its intention to acquire Splunk. This acquisition will integrate Splunk’s expertise in areas such as security information and event management, ransomware tools, industrial IoT vulnerability alerting, user behavior analytics and orchestration and digital experience monitoring that includes visibility into the performance of the underlying infrastructure.

Key Changes for Database Technology

Advancements in AI and ML integration are making database technology more intuitive and efficient. Oracle Database 23c features AI Vector Search, which simplifies interactions with data by using ML to identify similar objects in datasets. Oracle also introduced the Fusion Data Intelligence Platform, which combines data, analytics, AI models and apps to provide a comprehensive view of various business aspects. The platform also employs AI/ML models to automate tasks including data categorization, anomaly detection, predictive analytics for forecasting and customer segmentation, workflow optimization and robotic process automation.

In my previous discussion about IBM’s partnership with AWS, a major highlight is the integration of Amazon Relational Database Service with IBM Db2. This collaboration brings a fully managed Db2 database engine to AWS’s infrastructure, offering scalability and various storage options. The partnership between AWS and IBM will likely grow as the trend of companies forming more integrated and significant ecosystems continues.

Database technology also evolved with MongoDB queryable encryption features for continuous data content concealment. MongoDB Atlas Vector Search now also integrates with Amazon Bedrock, which enables developers to deploy generative AI applications on AWS more effectively. It’s also notable that Couchbase announced Capella iQ, which integrates generative AI technologies that exploit natural language processing to automatically create sample code, data sets and even unit tests. By doing this, the tool is streamlining the development process, enabling developers to focus more on high-level tasks rather than the nitty-gritty of code writing.

Leveraging Data Integration Platforms

Generative AI technologies have also improved data integration capabilities by using historical data, analyses of trends, customer behaviors and market dynamics. This advancement is particularly influential in the finance, retail and healthcare sectors, where predictive insights are critical for strategic and operational decisions. There’s been a shift towards adopting data lake house architectures, which combine the features of data lakes and data warehouses to help meet the challenges of handling large, varied data types and formats, providing both scalability and efficient management. This evolution in data architecture caters to the growing complexity and volume of data in various industries.

Integrating various data sources is crucial for many companies to enhance their business operations. Software AG has introduced Super iPaaS, an evolution of the traditional integration platform as a service (iPaaS). This advanced platform is AI-enabled and designed to integrate hybrid environments, offering expansive integration capabilities. Cloudera has also made strides with new data management features that incorporate generative AI, enabling the use of unstructured data both on-premises and in cloud environments. Its hybrid approach effectively consolidates client data for better management. Informatica’s intelligent data management cloud platform integrates AI and automation tools, streamlining the process of collecting, integrating, cleaning and analyzing data from diverse sources and formats. This creates an accessible data repository that benefits business intelligence and analytics.

That’s a Wrap!

In my collaborations throughout the year with various companies, one key theme has emerged in this AI-driven era – data has become even more fundamentally important for businesses. It’s clear that the success of AI heavily relies on the quality of the data it uses, and AI models are effective only when the data they process is accurate, relevant and unbiased.

For example, in applications such as CRM or supply chain optimization, outcomes are directly influenced by the data’s integrity. Instances where AI failed to meet expectations could often be traced to poor data quality, whether it was incomplete, outdated or biased. This year has highlighted the necessity of not just collecting large amounts of data but ensuring its quality and relevance. Real-world experience underscores the need for strict data governance and the implementation of systems that guarantee data accuracy and fairness, all of which are essential for the effective use of AI in business.

As AI technology advances and data quality improves, the use of generative AI in understanding and engaging with customers is becoming ever more prominent. Backed by good data management, this enhances the customer experience by making the customer journey more personalized and informative. It allows businesses to gain valuable insights from customer interactions, helping them continuously refine and improve their offerings and customer relations. I expect this trend to grow, further emphasizing the role of AI in customer engagement and shaping business strategies. In fact, this symbiotic relationship between AI-driven personalization and customer engagement is becoming a cornerstone of not only data management strategy but modern business strategy overall, significantly impacting how companies connect with their customers.

Wrapping up, it’s evident that the emphasis on data quality is critical for improving AI’s performance. Data management, cloud services, data protection and governance, databases, data integration and intelligent platforms have all significantly contributed to the advancement of AI. In 2024, I expect we’ll see even more emphasis on ensuring the accuracy and relevance of data so that AI can provide dependable insights.