Cover

Ready reference for MLS-C01 and SAP-C01: Amazon Kinesis Streams Compared

No need to study and find out inputs and outputs supported by Kinesis Streams by digging into AWS Documentation
Here is a ready to refer comparision of Kinesis streams available on AWS
Score points in exam:
- Know the right sources and sinks that each stream supports
- Be cautious about tricky exam questions that state a wrong answer option involving kinesis streams

Kinesis Video Streams - KVS

Though the name is Kinesis Video Streams, it supports ingestion of any kind of time series or time encoded data
Video:
- Stream video frames to KVS for storage, analytics, machine learning, playback or custom video processing
- Kinesis Video Streams provides fully-managed capabilities for HTTP Live Streaming (HLS) and Dynamic Adaptive Streaming over HTTP (DASH).
- Kinesis Video Streams also supports ultra-low latency two-way media streaming with WebRTC, as a fully managed capability
Time Encoded Data:
- Definition: Time-encoded data is any data in which the records are in a time series, and each record is related to its previous and next records e.g. video
- Non video examples, scientific applications:
  - LIDAR: LIght Detection And Ranging, is a remote sensing method that uses light in the form of a pulsed laser to measure ranges (variable distances) to the Earth
  - RADAR: RAdio Detection And Ranging, is a detection system that uses radio waves to determine the range, angle, or velocity of objects. It can be used to detect aircraft, ships, spacecraft etc.
  - Audio: any audio data
Write using:
- PutMedia API
- Producer SDK
Read using:
- GetMedia API
- Kinesis Video Streams Parser Library which can parse output from GetMedia API

Write clickstreams, application logs, social media and thousands of sources to Amazon Kinesis data stream
Write data to KDS using:
- Kinesis Producer Library that manages concurrency and generates high throughput within limits of Kinesis Data Streams
- Use Kinesis API like PutRecords() from any compute e.g. Lambda, EC2 etc. and write to KDS
- Using kinesis agent
Read using:
- Kinesis Client Library, from any app
- GetRecords() API
- Subscribe a Lambda where KDS is event source for a lambda

KDF is mainly meant only for data delivery unlike other streams
It delivers realtime data from a source to various destinations with a transformation applied if need be
Built-in support for JSON data to be written to Apache Parquet format to S3, no need of coding
- Any other format, other than JSON, needs coding e.g. CSV
- These are called as lambda blueprints
Refer diagram below, to know data delivery destinations supported
You can add data to an Amazon Kinesis Data Firehose delivery stream through Amazon Kinesis Agent or Firehose’s PutRecord and PutRecordBatch operations
Kinesis Data Firehose is also integrated with other AWS data sources such as Kinesis Data Streams, AWS IoT, Amazon CloudWatch Logs, and Amazon CloudWatch Events
Can write to firehose using kinesis agent
- After installing Amazon Kinesis Agent on your servers, you can configure it to monitor certain files on the disk and then continuously send new data to your delivery stream. For more information, see Writing with Agents

Realtime data analytics on streaming data using SQL
Apply ML algorithms and perform anomaly detection
- Anomaly detection is just an inbuilt SQL function that internally uses RCF - Random Cut Forest algorithm
Use Apache Flink applications in a supported language such as Java or Scala using built-in templates and operators for common processing functions to organize, transform, aggregate, and analyze data at any scale
Remember: KDA Cannot directly write to S3, attach a KDF i.e firehose to deliver KDA data to S3

Leave your comments to appreciate the article or request for change. Thanks!