Source: healthitanalytics.com
The AWS Analytics Lens is part of the AWS Well-Architected Framework (WAF) meant to sync hyper-performance IT, Cloud and Big Data Analytics.
In May 2020, Amazon Web Services (AWS) announced the launch of its latest Cloud Architect application for customized design, analytics and workloads within the AWS Cloud ecosystem. It’s called the Amazon Analytics Lens.
The AWS Analytics Lens is part of the AWS Well-Architected Framework (WAF). AWS WAF helps cloud architects and engineers work closely in sync to build highly secure, hyper-performance IT and Cloud infrastructure. These infrastructures are aimed at providing customers who want a resilient, cost-effective and Cloud management solutions and services with on-demand analytics and workload management features.
Modernization of Cloud systems has led to a massive shift in the way data engineers work with Big Data and analytics systems. AWS Analytics Lens makes the Cloud Modernization journey easy to manage, providing purpose-built tools specifically tailored for various industries, applications and scenarios arising from complex conditions.
Let’s define what is Amazon Analytics Lens?
In 2017, AWS had decided to add the Lens concept to its WAF system.
AWS WAF is built on the 5 pillars. These are:
- Ops Excellence: Automation, Customer Experience, and Real-time monitoring
- Security: Data management, governance, and compliance
- Reliability: Risk and recovery management, workload distribution, etc.
- Performance efficiency: Customized workload management based on predictive intelligence analytics
- Cost Optimization
Amazon Web Services Analytics Lens carries forward the legacy of AWS Well-Architected to new industry domains such as Robotic Process Automation (RPA), AI and Machine Learning, Serverless systems, Analytics, HPC, IoT, Blockchain, and Fintech.
Understanding Amazon Analytics Lens for Core Data Science Projects
AWS engineers can fully control the various layers of Cloud modernization process, powered by AWS Well-Architected Framework concepts.
The Analytics Lens works in layers and finally connects to all these a central repository of data, called the Data Lake.
These layers are mentioned as below:
Data Ingestion
This layer comprises of a family of services that ingest real time data and streamline ETL workloads on the Amazon Simple Storage Service (Amazon S3).
Components in this layer include:
- Amazon S3
- Amazon MSK
- Data Base Migration Services
- AWS Snowball – to accelerate Amazon DMS migrations into and out of AWS Cloud
- AWS Direct Connect – to secure connection between data center and AWS Cloud
Data Routing and On-Premise Storage
- AWS IoT Core – to route message to AWS data stores
- AWS DataSync – to automate data transfer between on-premises storage systems
- Storage systems like Amazon EFS, Amazon S3 and NFS are part of this layer.
Cloud Data Security, and Access Management
Security and access management have become the mainstay of modern IT and Cloud migration systems. AS Well-Architected Framework extensively focuses on the various mechanisms that help to secure data access within the central data lakes and data catalog with better monitoring, reporting and encryption features.
Components include:
- AWS Identity and Access Management (IAM)
- AWS CloudTrail
- AWS CloudWatch
- AWS Key Management Service (KMS)
- AWS CloudHSM (Hardware Security Module)
Integrated Data Lake Service
Analytics Lens also makes it easy to fully understand and differentiate between the layers. For example, it is capable of managing AWS Lake Formation and make it available for complex AI and Machine Learning projects.
It augments the AWS IAM and KMS modules, enabling granular access to data stored in the Data Lakes through a simple access management mechanism.
Catalog and Search
When Big Data workloads increase, Cloud architects need better control over analytics and search data operations. This is achieved using AWS ETL and metadata catalogs.
Components include:
- AWS Glue Data Catalog
- Amazon Elasticsearch Service
- Amazon RDS – to create Hive metastore for EMR clusters
- Amazon DynamoDB
Central Storage Layer
- Amazon S3
- Amazon EBS
Data Processing and Analytics
Amazon EMR – to scale Apache Spark, Hadoop, HBase, Presto, Hive, and Big Data frameworks available from open source and commerce data warehouses
Amazon Redshift / Redshift Spectrum
User Access and Interface
- AWS Lambda
- Amazon API Gateway
- Amazon Cognito
AWS Well-Architected Framework can also be understood from this graphical representation, created by the Amazon Web Services.
Applications of the Amazon Analytics Lens
Analytics Lens by AWS can be applied to a large number of scenarios within the data science domains, including On-prem Cloud and Edge computing.
Some of the common scenarios where Lens and Well-Architected Framework can be applied are listed as below:
- Building customized Data Lakes for your various AI and Machine Learning projects
- Create hyper-relevant Analytics dashboards to meet growing data science demands of data scientists, AI engineers, develops, AutoML and BI / BA teams.
- To integrate data streams from Open source frameworks – Hadoop, Presto, and Spark, in addition to also provide integrations from commercial warehouses.
- Improve R&D innovations with high-performing, superfast Cloud systems, powered by AI and Machine Learning algorithms within AWS Cloud.
- Superior handling of structured and unstructured Big Data, with smoother data preparation operations.
There are many more scenarios where AWS WAF and Lens could fit and diversify your Big Data Management and Analytics goals.
Dynatrace Announces Enhanced AI-powered Observability for Microsoft Azure
We shall keep updating the AWS Analytics Lens and AWS WAF resources in the coming weeks.