-
Characteristics
- Decouples applications for independent scaling
- Delivery of messages at least once, at low latency
- Suitable for data arriving at high, unpredictable rates
- Natural pairing with Dataflow for analyzing streaming data
- Enables independent applications to send and receive messages
- Holds onto data for up to seven days
- Does not guarantee first in, first out
- Can be the single ingest for all data points
- Can dynamically rate limit, enabling users to throttle messages
- Provides end-to-end reliability
- Acknowledges at least one guaranteed delivery and receipt of each individual message
- There is a small chance some messages might be delivered more than once
- Messages are Published to a specific topic
- Messages do not have to be received synchronously
- Messages are completely encrypted
- Applications can publish messages and one or more subscribers receive them
- Decouples sender's transmission of the message from the receiver's receipt of the message
- Message arrives at the subscriber where it is consumed
- After a message is consumed, the publisher sends an acknowledgment to the subscription
- When all deliveries are complete, messages are removed from the queue
- No clusters to configure. Users simply pay for what they use
- Uses two levels of indirection between the publisher and the subscriber
- Offers on demand scalability to one million messages per second and beyond
- PubSub builds on the same technology Google uses internally
-
Concepts
- A Message represents data in transit through a system
- A Message consists of a payload and optional attributes that describe the payload
- Data is transmitted to one or more subscriptions
- A Topic is a feed of data
- Subscribing Forwarder ensures data delivery
- Subscriptions expresses an interest in receiving messages
- Subscriptions register each delivery of data
- Subscription determines which subscribers are registered to receive a message and queues up the messages to be sent
- Subscribers receive messages by Pub/Sub, pushing them to the endpoint or by pulling them from the service
- Subscribers can be configured to receive messages on a push or pull basis
- Publishing Forwarder receives data from the publisher on behalf of the topic
- Pull subscribers use HTTPS requests to google APIs
- Push subscribers can accept post requests over HTTPS
- Push subscribers use webhook endpoints
- Pub is short for publishers
- Sub is short for subscribers
-
Use cases
- Balancing workloads in network clusters
- Implementing asynchronous workflows
- Aggregating log data from multiple systems
- Refreshing distributed caches
- Distributing event notifications
- Data streaming from various processes or devices
- Reliability improvement
-
Use case scenarios
- A large queue of tasks can be efficiently distributed among multiple workers
- An order processing application can place an order on a topic and processed by one or more workers
- A service that accepts user sign ups can send notifications whenever a new user registers
- An application can publish invalidation events to update the IDs of objects that have changed
- A Google Compute Engine instance can write logs to the monitoring system to a database for later querying
- A residential sensor can stream data to back-end services hosted in the cloud
- A single zone Compute Engine service can operate in additional zones by subscribing to a common topic to recover from failures in a zone or region