Communication is key within enterprise IT, especially in the advent of public cloud. IT administrators need a way...
to reach out to end users, alert in-house staff or enable greater communication across different types of software. But no single service can handle all those disparate tasks.
Messaging services allow software programs to communicate among distributed systems, to isolate resources and interdependencies, and to connect environments with different languages, compilers and operating systems. There are different types of messaging modes and patterns based on application-specific requirements. AWS messaging tools and services, such as Amazon Simple Queue Service (SQS), Amazon Simple Notification Service (SNS), Amazon Kinesis and AWS Lambda, support these requirements.
Different messaging types
Messaging systems can be categorized into two groups: synchronous and asynchronous. With synchronous messaging, the client sends a message and waits for a response from the recipient before sending the next one. Conversely, with asynchronous messaging, the sender doesn't wait for a response.
Both systems handle push-pull, publish-subscribe and request-response messaging patterns. Each of these messaging patterns was developed to serve a different purpose.
Push-pull is one-way communication that enables a consumer to distribute a message -- push -- to all clients evenly and queue messages -- pull -- in the different clients' pipelines. The push-pull procedure is commonly used in email exchange and file-processing jobs such as video recompressing and image scaling.
In a publish-subscribe messaging pattern, the message is sent to characterized channels or named queues, rather than to a particular recipient. Therefore, a message publisher doesn't know the specifics of its subscribers. The subscriber can then receive the message without having to inform the publisher.
Request-response or request-reply is the most common messaging pattern. In this message exchange, a requester sends a request message to a system. The system then processes the message, gets the request and returns a response. This synchronous messaging type enables seamless communications in client-server architectures.
Putting AWS messaging into practice
Administrators use publish-subscribe messaging to notify IT staff about important events or critical errors. Amazon SNS uses the publish-subscribe pattern -- posting a message without prior knowledge of the customers or subscribers. The message posts to a Topic, which customers can subscribe to for real-time notifications. Amazon SNS delivers notifications using a push mechanism, which eliminates the need to regularly check or poll for new information and updates.
Push-pull messaging helps developers build a decoupled system. For example, an end user uploads a video and different processes, then transcodes and prepares the video that will transmit over the internet.
Amazon SQS provides queues where one or more consumers can push messages, while other consumers can poll the queue and process messages. SQS is an at-least-once queue, which means that it won't deliver the same message to another customer after it has been read for a VisibilityTimeout period. If the consumer doesn't delete the message within that time, Amazon SQS assumes something went wrong and redelivers it to another consumer. The consumer has to delete the message after the process completes or it will show up again in the queue.
SNS vs. SQS
Amazon SNS pushes message to subscribers, whereas SQS is a distributed AWS messaging system in which receivers have to poll the service to receive messages. Multiple recipients can't obtain a message; only one receiver can receive, process and delete it.
Polling in SQS has some latency, but SNS messages immediately push to subscriber end points. IT teams mainly use SQS to decouple or integrate applications, and messages only store for a short time. In a real-world scenario, if an IT team wants to replicate application data to several storage systems, SNS can send this data to multiple subscribers and replicate the message it receives on different storage systems.
Additional AWS messaging options
Even though Amazon Kinesis and AWS Lambda serve other purposes, the services can be used as message queues to support decoupled and microservices-based applications.
Developers can use Amazon Kinesis as an asynchronous real-time AWS messaging tool that streams messages received from producers. Consumers then read and process those messages in the order they are received without sending confirmation to the producer.
IT teams can also use AWS Lambda with Amazon API Gateway to run code without provisioning or managing servers. Code instead runs in response to events, such as changing data in Simple Storage Service or a DynamoDB table, invoking code using API calls through SDKs and running code in response to HTTP requests via the Amazon API Gateway. This creates asynchronous request-response messaging sent over HTTP -- this channel stays open until a response is received. This HTTP request begins with Amazon API Gateway and then Amazon Lambda runs a process on the system and returns the response.
For less-experienced IT pros, SNS is the best option for sending basic messages. But, in most cases, IT teams should use SQS for message queue management.
Test your Amazon SQS knowledge
Workarounds for AWS Lambda from SQS queue functions
Use message queues to decouple application functions