Instrumentation
Stackdriver
Monitoring
Logging
Trace
Error Reporting
Debugging
Profiler
Storage
Bigtable
Features
Single lookup key
Large amoutns of data
Hight throuput
Same as HBase
Can be ported
High Scalability
Data encryption
In transit
At rest
Streaming
Flow streaming
Spark
Batch
Hadoop
Spark
Used by google
Maps
Gmail
Use Cases
IoT
User analytics
Finances
No-SQL
Cloud Storage
Object Storage
Consists of
Buckets
Name
Geography
Geo location per user
Storage Class
Access
Control with IAM
ACLs possible
Scope
Who
Action
Permissions
Can be versioned
New > Old
Features
Immutable
Server side encryption
Data in Transit -> HTTPS
Lifecycle management policy
Each object has a URL
Bring Data
Online Transfer
Storage Transfer (online)
Transfer appliance (offline)
1PTB / appliance
Still beta
Interoperability
Bog Query
Imp/Exp tables
Compute Engine
Scripts
Images
General objects
Cloud SQL
Tables
App Engine
Object
Logs
Backups
Cloud SQL
Databases
As a service ->
MySQL
PostgreSQL
Beta
IaaS
SQL in VM possible
Backups
Security
Firewalls
Encrypted data at rest
Accesible by services
Cloud Spanner
Features
Horizontal scalability
Global consistency
Auto replication
Huge capacity
Use Cases
Huge sharded databases
Need global consistency
Database consolidation
FInances
Inventory uses
Cloud Datastore
Use Cases
Structure Data
Engine apps
Integration between ->
App engine
Compute engine
Features
Sharding
Transactionality
SQL like queries
Free daily quota
CRUD
Also No-SQL
Options comparison
Type
No SQL Document
NoSQL Wide column
BlobStore
Relational SQL for OLTP
Relational SQL for OLTP
Relational SQL for OLAP
Transactions
Yes
Single-row
No
Yes
Yes
No
Complex Queries
No
No
No
Yes
Yes
Yes
Capacity
Terabytes+
Petabytes+
Petabytes+
Terabytes
Petabytes
Petabytes+
Unit Size
1MB/entity
5TB/object
~10 MB/cell ~ 100 MB/row
Determined by DB engine
10,240 MiB/row
10 MB/row
Organize
Project
Group related resources with common business bojecive
Interact
PC Console
Cloud Shell and Cloud SDK
Cloud Console in Mobile App
REST API
Machine Learning
Natural Language API
Convert audio to text
80 languages now
Syntac analysis
Entity recognition
flagging
Vision API
Content of image
Machine Learning
Use cases
Structured Date
Clasification
Regresion
Recommendation engine
Fraud detection
Unstructured
Image & vid analysis
Text analytics
Speech API
Translate API
Tensor Flow
Developed by google
Video intelligence API
Video content search and discover
Definition
Resource pooling
Rapid elasticity
Measured Service
Broad network access
On-Demand Self-Service
Compute
Compute Engine
VPC
VMs
Networking
Connectivity options
Kubernetes Engine
Offering
IaaS
PaaS
Containers
Pod
=
Process
App Engine
Environments
Standard
Features
Autoscale
Free daily quota
Usage based pricing
Simple
SDK
Runtimes
Not supported
Java
Python
PHP
GO
Supported
.NET
Runs in sandbox
Constraints
No writing in local files
Requests time out after 60 sec
Limited 3rd party software use
Flexible
Containers
No Sandbox
Pay allocation/hour
Cloud Funcitons
100 ms intervals
Same as Azure Functions
Only JavaScript possible
Options Comparison
Big Data
Big Query
Data explodation
Massive SQL queries
Ingress
Cloud Storage
Cloud DataStore
Stream 100rows/sec
Billing
Free monthly quotas
Separate for data store
Pay only when query runs
If > 90 days old price down
Presence
Global
Data in ->
US
EU
Asia
Access
Can share dataset
Pub/Sub
Stream analytics
Messaging platform
Quarantees
At least once delivery
1 mil msg/sec
Dataflow
Features
ETL
Batch
Continues comp
Auto-scaling
Data pipelines
Use Cases
Fraud detection
Finances
IoT
Healthcare
Logistics
Streaming
POS
Real time apps
Dataproc
Apache
Spark
Hive
Pig
Billing
Per sec
1 min minimum
Use Preemptble VMs
Data mining
Use cases
Predictable size
Datalab
Jupyter notebooks
Only python
Fully integrated