Cloud & MLOps ☁️
Amazon QuickSight

Amazon QuickSight

  • Business analytics and visualizations in the cloud

  • Fast, easy, cloud-powered business analytics service

  • Allows all employees in an organization to:

    • Build visualizations off of data

    • Geared towards a more general audience and not so much for developers

    • Perform ad-hoc analysis on datasets

    • Quickly get business insights from data

    • Anytime, on any device (browsers, mobile)

  • Serverless

    • Analysts don't need to manage their own servers.

QuickSight Data Sources

Quick sight can connect to a wide variety of data sources,

  • Redshift Data warehouse

  • Aurora / RDS

  • Athena

  • EC2-hosted databases

  • Files (S3 or on-premises)

    • Excel

    • CSV, TSV

    • Common or extended log format

  • Data preparation allows some limited ETL

    • E.g., changing field names, data types, some calculated columns etc.

SPICE - Making QuickSight Fast

  • Data sets are imported into SPICE (Super-fast, Parallel, In-memory Calculation Engine)

    • Proprietary thing that uses columnar storage and in memory processing and machine code generation to make quick sight extremely fast

    • Uses columnar storage, in-memory, machine code generation

    • Accelerates interactive queries on large datasets

  • Each user gets 10GB of SPICE

  • Highly available / durable

  • Scales to hundreds of thousands of users

QuickSight Use Cases

  • Interactive ad-hoc exploration / visualization of data

  • Dashboards and KPI dashboards

  • Stories

    • Guided tours through specific views of an analysis

    • Convey key points, thought process, or the evolution of an analysis in a narrative form

  • Analyze / visualize data from a variety of sources:

    • Logs in S3

    • On-premise databases

    • AWS (RDS, Redshift, Athena, S3)

    • SaaS applications, such as Salesforce

    • Any JDBC/ODBC data source endpoint

Machine Learning Insights

3 major features:

  1. ML powered Anomaly detection

    • Amazon quick sight is using the random cut forest algorithm, that Amazon developed, to analyze millions/billions of data points

    • Rapidly detects the outliers

  2. Forecasting

    • enables non-technical users to confidently forecast their key business metrics

    • random cut forest algorithm

    • Detects seasonality and trends

  3. Auto-narratives

    • Builds a dashboards automatically in plain language

    • just a way of translating the trends and seasonality in your data into words that you can actually put into a report

What not to use QuickSight For

  • Highly formatted canned reports

    • QuickSight is for ad-hoc queries, analysis, and visualization
  • ETL

    • Use Glue instead, although QuickSight can do some limited transformations

QuickSight Security

  • Multi-factor authentication on your account

  • VPC connectivity

    • Add QuickSight's IP address range to your database security groups
  • Row-level security

  • Private VPC access

    • Use Elastic Network Interface or AWS Direct Connect to enable those capabilites

QuickSight User Management

  • Users defined via IAM, or an email signup

  • Can integrate with Active Directory with QuickSight Enterprise Edition

    • Active Directory (AD) is Microsoft's proprietary directory service. It runs on Windows Server and enables administrators to manage permissions and access to network resources.

    • Allows IT departments to manage and store information about the devices, users, and objects within your organizations network.

QuickSight Pricing

  • Annual subscription

    • Standard: $9 / user /month

    • Enterprise: $18 / user / month

  • Can pay Month to month

    • Standard: $12 / GB / month

    • Enterprise: $24 / GB / month

  • Can purchase extra SPICE capacity (beyond 10GB)

    • $0.25 (standard) $0.38 (enterprise) / GB / month
  • Enterprise edition

    • Encryption at rest

    • Microsoft Active Directory integration

QuickSight Dashboards

https://d2908q01vomqb2.cloudfront.net/b6692ea5df920cad691c20319a6fffd7a4a766b8/2019/02/01/InteractiveQuicksightDashboards_rev.png

Dashboards are a read only snapshot of an analysis that you've created. Once you've created a dashboard, you can share that dashboard with other users who have access to QuickSight, but they cannot edit or change those filters. It's just a collection of various charts and graphs of relevant performance indicators that they care about.

QuickSight Visual Types

  • AutoGraph

    • Just automatically selects the most appropriate visualization based on the properties of the data itself, instead of making you select one yourself

    • these visualization types are chosen to best reveal the data in relationships in an effective way for you

  • Bar Charts

    • For comparison and distribution (histograms)
  • Line graphs

    • For changes over time

    • Looking for trends or seasonality

  • Scatter plots, heat maps

    • For correlation
  • Pie graphs

    • For aggregation
  • Tree maps

    • Heirarchical Aggregation

https://upload.wikimedia.org/wikipedia/commons/d/d8/Benin_English.png

  • Pie charts within pie charts. Each section is further broken down into sub-classifications.

  • Pivot tables

    • For tabular data

    • Organizing and aggregating in different ways

  • Stories

    • narratives that present iterations of your data