Pulumi

When working in cloud environments, it is useful to be able to define and statefully manage infrastructure. Pulumi is an Infrastructure as Code (IAC) utility that allows developers to programatically create, update, and delete cloud resources using programming languages such as Typescript, Javascript, Python, and Go.

Setup

1. Download and install Pulumi following the instructions here.

2. Confirm the CLI is successfully installed by running the following.

   CLI

   Command

   pulumi version
   

   Example Output

   v3.39.1
   

3. Within an empty directory, execute one of the following to create a Pulumi project for the target programming language.

   Typescript Javascript Python Go

   pulumi new aws-typescript
   

   pulumi new aws-javascript
   

   pulumi new aws-python
   

   pulumi new aws-go
   

4. Once the Pulumi project has initialized, run the following command to ensure Pulumi is able to execute the project for the target language. The pulumi preview command will list which cloud resources will be generated by the Pulumi program. For a default Pulumi project, there should only be the default S3 bucket for data storage.

   CLI

   Command

   pulumi preview
   

   Example Output

   Previewing update (dev)
   
   View Live: https://app.pulumi.com/username/test/dev/previews/5aa60450-112f-4394-99c7-233c13822001
   
        Type                 Name       Plan
    +   pulumi:pulumi:Stack  test-dev   create
    +   └─ aws:s3:Bucket     my-bucket  create
   
   Outputs:
       bucketName: output<string>
   
   Resources:
       + 2 to create
   

5. To verify deployments are working as expected, execute the following:

   CLI

   Command

   pulumi up -f
   

   Example Output

   Updating (dev)
   
   View Live: https://app.pulumi.com/username/test/dev/updates/1
   
        Type                 Name       Status
    +   pulumi:pulumi:Stack  test-dev   created
    +   └─ aws:s3:Bucket     my-bucket  created
   
   Outputs:
       bucketName: "my-bucket-9f5953e"
   
   Resources:
       + 2 created
   
   Duration: 6s
   

6. Once the above runs successfully, run the following to delete any provisioned resources.

   CLI

   Command

   pulumi down -f
   

   Example Output

   Destroying (dev)
   
   View Live: https://app.pulumi.com/username/test/dev/updates/2
   
        Type                 Name       Status
    -   pulumi:pulumi:Stack  test-dev   deleted
    -   └─ aws:s3:Bucket     my-bucket  deleted
   
   Outputs:
     - bucketName: "my-bucket-fb2b85f"
   
   Resources:
       - 2 deleted
   
   Duration: 4s
   

LocalStack

While it is preferrable to develop and test applications against actual cloud resources when possible, not all readers may have AWS access or be able to set up a private account. Additionally, there is always the risk of leaving cloud resources provisioned, which may result in unintended charges. To make these tutorials as accessible and cost-effective as possible, examples in these tutorials leverage the free-tier version of LocalStack whenever possible.

LocalStack is able to emulate an AWS cloud environment on the user’s development machine, making it useful for both development and testing. Examples that work and function with the free-tier version of LocalStack will be appropriately marked.

Dependencies

LocalStack requires the following be installed on the user’s local machine to function properly.

• Python & PIP
• Docker Desktop

Navigate to the Python and Docker Desktop installation page for platform-specific setup instruction.

LocalStack CLI

LocalStack offers a CLI for provisioning an AWS test environment. To install it, execute the following.

pip3 install localstack

Initialize Localstack with the following:

localstack start

AWS Local Client

AWS CLI

Adding the --endpoint-url option to the AWS CLI will direct API requests to the Localstack instance. For example, the following will create an AWS S3 data storage bucket within Localstack.

aws --endpoint-url=http://localhost:4566 s3 mb s3://mytestbucket

Accounts

After initially signing up for AWS, a default root account is created. AWS accounts function as containers for organizing and isolating cloud resources. For example, deployment environments, such as development, staging, and production, often utilize distinct AWS accounts. In addition, accounts act as a security boundary, ensuring only authorized users and systems can access particular cloud resources. ¹

An AWS account has the following unique identifiers:²

• AWS Account ID: 12-digit unique ID
• Canonical User ID: Obfuscated form of the account ID. Used when granting cross-account access to cloud resources.

The active account ID can be fetched from the Security Token Service (STS) with the following CLI command:

aws sts get-caller-identity --query Account --output text

123456789012

The command aws sts get-caller-identity fetches the active user information leveraged by the CLI, which includes the user ID, the account ID, and the user access resource number (ARN). The --query flag enables users to target a particular field to output in the response and --output specifies the desired format (yaml, josn, text, etc.) ^{3 4}

The simplest way to fetch the canonical ID is via the Simple Storage Service (S3) API CLI command.

aws s3api list-buckets --query Owner.ID --output text

79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2be

Regions and Availability Zones

Regions

An AWS region is a geographic area that contains interconnected AWS data centers used for provisioning cloud resources. Available regions can be listed with the following CLI command:

aws ec2 describe-regions --all-regions --query  Regions[*].RegionName

[
    "af-south-1",
    "eu-north-1",
    "ap-south-1",
    ...
]

Regions are isolated from one another for additional fault tolerance and stability.¹ AWS cloud service offerings may differ between regions. Therefore, cloud engineers should verify that any required cloud services are available in the desired regions before beginning development.

Availability Zones

An availabiilty zone (commonly abreviated as AZ) is one or more discrete AWS data centers that have redundant connectivity, networking, and power within a given AWS region. AZs are interconnected through low-latency networking and data replication. It is considered best practice to deploy cloud applications across multiple AZs for increased fault tolerance, in the event that one or more AZs experience technical outages.²

AZ names are of the following format \<region-name\>\<letter[a-z]\>. For example:.

aws ec2 describe-availability-zones --query AvailabilityZones[*].ZoneName --region us-east-2

[
    "us-east-2a",
    "us-east-2b",
    "us-east-2c"
]

Resources

A resource is a broad term for any cloud entity that can be provisioned in AWS. For instance, servers, virtual private networks, networking policies, and account users are considered AWS resources. Every resource is associated with an Amazon Resource Number (ARN), which uniquely identifies it. ARNs have the following format¹:

arn:aws:[service]:[region]:[account-id]:[resource-id]
arn:aws:[service]:[region]:[account-id]:[resource-type]:[resource-id]
arn:aws:[service]:[region]:[account-id]:[resource-type]/[resource-id]

A breakdown of the above fields is provided below:

• service: The AWS service which the resource is associated with
• region: Region in which the resource is located.
• account-id: The account which contains the resource
• resource-type: The type of service resource (i.e. users, compute servers, managed databases, etc.)
• resource-id: The unique resource identifier

For instance, the following fetches the ARN of the active AWS user associated with the local developers AWS credentials.

aws sts get-caller-identity --query Arn --output text --region us-east1

arn:aws:iam::012345678910:user/username

For the above output, iam refers to the AWS Identity Access Management service. This is followed by the account number which owns the user entity and the IAM resource is of type user.

Tags

Tags are user-defined metadata that can be attached to resources. This can be used to distinguish and group resources.

For instance, to add a tag to your active user.

export USERNAME=$(aws iam get-user --query  User.UserName)
aws iam tag-user --user-name $USERNAME --tags '{"Tag": "You are it!"}'
aws iam list-user-tags --user-name $USERNAME

{
    "Tags": [
        {
            "Tag": "You are it!"
        }
    ]
}

Services

AWS cloud resources are available through services, APIs accessible through the AWS console or programtically. As an introduction, common service categories and offerings are briefly explored here.^{1 2}

Compute

Compute services enable users to run and host programs and applications.

Name	Logo	Description
EC2		Elastic Cloud Compute: Provisioning and managing of virtual and private physical servers
ECS		Elastic Container Service: Executing containerized applications on custom infrastructure
EKS		Elastic Kubernetes Service: Managed Kubernetes clusters
Lambda		Code execution in ephemeral environments without need for provisioning/managing underlying infrastructure

Storage

AWS offers various data storage services of the following types:

• Object Storage: Stores objects, composed of data and user metadata.
• Block Storage: Data is stored within a block of memory³
• File Storage: Storage provided via a file system.

Name	Logo	Description
S3		Simple Storage Service. Object storage where data can be uploaded to uniquely named data buckets under a unique key
EBS		Elastic Block Storage. Provides SSD and HDD block storage for EC2 servers
EFS		Elastic File Storage. Serverless remote storage via the NFSv4 protocol

Database

While databases can be configured by leveraging both compute and storage services, AWS offers database services to facilitate proviioning, managing, and monitoring such systems. These offerings include the following database types:

• In-Memory: Intended to store frequently accessed values for increased read-performance.
• Relational: Tabular row data, similar to data stored in spreadsheets. Data typically well-defined.
• NoSQL: Data stored in documents (key-value pairs) that may not necessarily have a well-defined schema.
• Time Series: Data is indexed in such a manner so that is easy to query and analyze within a given date range.

Name	Logo	Description
Elasticache		Managed Redis and Memcached in-memory databases
RDS		Relational Database Service: For provisioning relational database systems (MySQL, Postgres, etc.)
DynamoDB		Key-value NoSQL database
DocumentDB		MongoDB-esque NoSQL database
Timestream		Enables querying for data within a certain date range.

Networking & Content Delivery

Networking & Content Delivery services allow cloud developers to define virtual networks, firewall rules, and CDNs to improve latency.

Name	Logo	Description
VPC		Define virtual private networks within a given IP range
Cloudfront		Managed CDN network for content delivery in different regions of the globe
Route53		Amazon’s DNS service

Security, Identity, & Compliance

Security, Identity, & Compliance services assist with securing and auditing access to both AWS account resources and cloud applications.

Name	Logo	Description
IAM		Ensures authorized access to AWS cloud resources
Cognito		Provides identity and login managmenet for cloud applications
Secret Manager		Manages storage and access of private application values (i.e. database credentials, private application keys, etc.)

Management and Governance

Management and Governance services are responsible for providing visibility into finacial, application, and user activity in the AWS cloud.

Name	Logo	Description
Cloudwatch		Application and service logging
Cloudtrail		Audit trail of cloud account activities

Introduction

AWS cloud services are available through public API endpoints. Whether invoked via the AWS CLI, SDKs, or Infrastructure as Code (IaC) tools, each approach ultimately results in sending HTTP requests to AWS service endpoints. These underlying HTTP requests are referred to as canonical requests.

AWS canonical requests include a signature generated with request parameters and AWS secret key. This signature enables AWS to validate the identity of the client, protect the API request data in transit, and mitigate potential relay attacks.

While developers may directly create, sign, and transmit canonical requests to AWS, it is often preferable to utilize the AWS-provided CLI and SDKs. Direct canonical calls to AWS are primarily recommended in cases when developing in unsupported programming languages or where fine-grained API control is required.

Request Structure

Canonical requests are composed of the following component:¹

CanonicalRequest =
    HTTPRequestMethod + '\n' +
    CanonicalURI + '\n' +
    CanonicalQueryString + '\n' +
    CanonicalHeaders + '\n' +
    SignedHeaders + '\n' +
    HexEncode(Hash(RequestPayload))

The individual components are defined below:

• HTTPRequestMethod: The HTTP operation
• CanonicalURI: Absolute path of the target resource, including the base service domain.
• CanonicalQueryString: URI-encoded query parameters
• CanonicalHeaders: List of all the HTTP headers included with the signed requests.
• SignedHeaders: Alphabetically sorted, semicolon-separated list of lowercase request header names
• RequestPayload: Payload of the target request. This is hashed and hex encoded for additional security.

Signing

All canonical requests include a hash value generated from the API parameters and account credentials. When AWS receives an API request, it generates the same hash from the provided API parameters and compares it to the original request to validate the identity of the caller. If the request hash and the AWS-generated hash do not match, the request is denied. The process of generating this request hash is known as signing.¹

Version 4 Signing

The version 4 signing process consists of the following steps and components²:

1. Creating the credential scope: This value restricts the request to the target service and region and is of the following format: TIMESTAMP/REGION/SERVICE/SIGNING_VERSION where the timestamp value is of form YYYYMMDD.

2. Generate the target string to sign: This consists of the signing algorithm used to produce the signature (AWS4-HMAC-SHA256), the Amzaon-formatted request timestamp (i.e. YYYYMMDDHHMMSSZ), the previously produced credential scope, and a hash of the canonical requests string, all separated by newline characters:

   Pseudocode

   signatureString = SIGNING_ALGORITHM + "\n" +
   AMAZON_DATE_TIMESTAMP + "\n" + 
   CREDENTIAL_SCOPE + "\n" + 
   SHA256(CANONICAL_REQUEST_STRING)
   

3. Create the signature key: The signature key, used to sign the request string, is derived from the AWS secret key, Amazon-formatted request timestamp, region, and service. The following Pseudocode illustrates this process:

   Pseudocode

   kDate = hash("AWS4" + Key, Date)
   kRegion = hash(kDate, Region)
   kService = hash(kRegion, Service)
   signatureKey = hash(kService, "aws4_request")
   

4. Sign the previously generated signature string with the signature key and encode the hexadecimal representation.

   Pseudocode

   signature = hexEncode(hash(signatureKey, signatureString))
   

Demo

Below provides a concrete example for generating a version 4 signature from an arbitrary string:

ts-node signing.ts $AWS_SECRET_KEY us-west-1 ssm "Hello World!"

// can_req/ts/signing.ts

import * as crypto from "crypto";

const SIGNING_ALGORITHM = "AWS4-HMAC-SHA256";

export function getTimestamps(): [string, string] {
  const now = new Date();
  const year = now.getUTCFullYear();
  const month = String(now.getUTCMonth() + 1).padStart(2, "0");
  const day = String(now.getUTCDate()).padStart(2, "0");
  const hours = String(now.getUTCHours()).padStart(2, "0");
  const minutes = String(now.getUTCMinutes()).padStart(2, "0");
  const seconds = String(now.getUTCSeconds()).padStart(2, "0");

  const amzTimestamp = `${year}${month}${day}T${hours}${minutes}${seconds}Z`;
  const reqTimestamp = `${year}${month}${day}`;
  return [amzTimestamp, reqTimestamp];
}

export function getCredentialScope(
  reqTimestamp: string,
  region: string,
  service: string
): string {
  return `${reqTimestamp}/${region}/${service}/aws4_request`;
}

export function getStringToSign(
  amzTimestamp: string,
  scope: string,
  message: string
): string {
  return [
    SIGNING_ALGORITHM,
    amzTimestamp,
    scope,
    computeSHA256SignatureHash(message),
  ].join("\n");
}

export function sign(key: Buffer, message: Buffer): Buffer {
  return crypto.createHmac("SHA256", key).update(message).digest();
}

export function signHex(key: Buffer, message: Buffer): string {
  return crypto.createHmac("SHA256", key).update(message).digest("hex");
}

export function computeSHA256SignatureHash(input: string): string {
  return crypto.createHash("SHA256").update(input).digest("hex");
}

export function getAWS4SignatureKey(
  key: string,
  reqTimestamp: string,
  region: string,
  service: string
): Buffer {
  const kDate = sign(Buffer.from("AWS4" + key), Buffer.from(reqTimestamp));
  const kRegion = sign(kDate, Buffer.from(region));
  const kService = sign(kRegion, Buffer.from(service));
  const kSigning = sign(kService, Buffer.from("aws4_request"));
  return kSigning;
}

if (require.main === module) {
  // Get user input
  const secretKey = process.argv[2];
  const region = process.argv[3];
  const service = process.argv[4];
  const userInput = process.argv[5];

  // Get the required timestamp strings
  let [amzTimestamp, reqTimestamp] = getTimestamps();
  console.log("Amazon Timestamp: " + amzTimestamp);
  console.log("Requset Timestamp: " + reqTimestamp);

  // Get the scope of the request (the timestamp and the target service)
  const scope = getCredentialScope(reqTimestamp, region, service);
  console.log("Credential Scope: " + scope);

  //  Get the AWS v4 signing key
  const key = getAWS4SignatureKey(secretKey, reqTimestamp, region, service);
  console.log("Signing Key: " + key.toString("hex"));

  // Prepare string value to sign from user input
  const stringToSign = getStringToSign(amzTimestamp, scope, userInput);
  console.log("String to sign: " + JSON.stringify(stringToSign));

  // Sign and output user string
  const signature = signHex(key, Buffer.from(stringToSign));
  console.log("Signed String: " + signature);
}

node signing.js $AWS_SECRET_KEY us-west-1 ssm "Hello World!"

// can_req/js/signing.js

var crypto = require("crypto");

const SIGNING_ALGORITHM = "AWS4-HMAC-SHA256";

function getTimestamps() {
  const now = new Date();
  const year = now.getUTCFullYear();
  const month = String(now.getUTCMonth() + 1).padStart(2, "0");
  const day = String(now.getUTCDate()).padStart(2, "0");
  const hours = String(now.getUTCHours()).padStart(2, "0");
  const minutes = String(now.getUTCMinutes()).padStart(2, "0");
  const seconds = String(now.getUTCSeconds()).padStart(2, "0");

  const amzTimestamp = `${year}${month}${day}T${hours}${minutes}${seconds}Z`;
  const reqTimestamp = `${year}${month}${day}`;
  return [amzTimestamp, reqTimestamp];
}

function getCredentialScope(reqTimestamp, region, service) {
  return `${reqTimestamp}/${region}/${service}/aws4_request`;
}

function getStringToSign(amzTimestamp, scope, message) {
  return [
    SIGNING_ALGORITHM,
    amzTimestamp,
    scope,
    computeSHA256SignatureHash(message),
  ].join("\n");
}

function sign(key, msg) {
  return crypto
    .createHmac("SHA256", key)
    .update(Buffer.from(msg, "utf-8"))
    .digest();
}

function signHex(key, msg) {
  return crypto.createHmac("SHA256", key).update(msg).digest("hex");
}

function computeSHA256SignatureHash(input) {
  return crypto
    .createHash("SHA256")
    .update(Buffer.from(input, "utf-8"))
    .digest("hex");
}

function getAWS4SignatureKey(key, reqTimestamp, region, service) {
  const kDate = sign(Buffer.from("AWS4" + key, "utf-8"), reqTimestamp);
  const kRegion = sign(kDate, region);
  const kService = sign(kRegion, service);
  const kSigning = sign(kService, "aws4_request");
  return kSigning;
}

if (require.main === module) {
  // Get user input
  const secretKey = process.argv[2];
  const region = process.argv[3];
  const service = process.argv[4];
  const userInput = process.argv[5];

  // Get the required timestamp strings
  [amzTimestamp, reqTimestamp] = getTimestamps();
  console.log("Amazon Timestamp: " + amzTimestamp);
  console.log("Request Timestamp: " + reqTimestamp);

  // Get the scope of the request (the timestamp and the target service)
  const scope = getCredentialScope(reqTimestamp, region, service);
  console.log("Credential Scope: " + scope);

  //  Get the AWS v4 signing key
  const key = getAWS4SignatureKey(secretKey, reqTimestamp, region, service);
  console.log("Signing Key: " + key.toString("hex"));

  // Prepare string value to sign from user input
  const stringToSign = getStringToSign(amzTimestamp, scope, userInput);
  console.log("String to sign: " + JSON.stringify(stringToSign));

  // Sign and output user string
  const signature = signHex(key, stringToSign);
  console.log("Signed String: " + signature);
}

module.exports = {
  getTimestamps,
  getCredentialScope,
  computeSHA256SignatureHash,
  getAWS4SignatureKey,
  getStringToSign,
  signHex,
};

python3 signing.py $AWS_SECRET_KEY us-west-1 ssm "Hello World!"

# can_req/py/signing.py

import base64, datetime, hashlib, hmac, json, sys

SIGNING_ALGORITHM = "AWS4-HMAC-SHA256"


def get_timestamps() -> tuple[str, str]:
    """Get strings of required timestamps for canonical requests"""
    now = datetime.datetime.utcnow()
    amazon_timestamp = now.strftime("%Y%m%dT%H%M%SZ")
    req_timestamp = now.strftime("%Y%m%d")
    return amazon_timestamp, req_timestamp


def get_credential_scope(req_timestamp: str, region: str, service: str) -> str:
    """Define the scope of the request, which includes the target region and service"""
    return "{}/{}/{}/aws4_request".format(req_timestamp, region, service)


def sign(key: str, msg: str) -> bytes:
    """Generate the HMAC-SHA256 hash of a target string using the provided secret key"""
    return hmac.new(key, msg.encode("utf-8"), hashlib.sha256).digest()


def compute_sha256_hash(input: str) -> str:
    """Create SHA256 hash of a target string"""
    m = hashlib.sha256()
    m.update(input.encode("utf-8"))
    result = m.hexdigest()
    return result


def get_string_to_sign(amzn_date_stamp: str, scope: str, can_req: str) -> str:
    """Get string to sign from request parameters"""
    return "\n".join(
        [SIGNING_ALGORITHM, amzn_date_stamp, scope, compute_sha256_hash(can_req)]
    )


def get_aws4_signature_key(
    key: str, datestamp: str, region: str, service_name: str
) -> bytes:
    """Generature canonical requests signature"""
    kdate = sign(("AWS4" + key).encode("utf-8"), datestamp)
    kregion = sign(kdate, region)
    kservice = sign(kregion, service_name)
    ksigning = sign(kservice, "aws4_request")
    return ksigning


if __name__ == "__main__":
    # Get user input from command args
    amazon_secret_key = sys.argv[1]
    region = sys.argv[2]
    service = sys.argv[3]
    user_input = sys.argv[4]

    # Fetch the required timestamps
    amazon_timestamp, req_timestamp = get_timestamps()
    print("Amazon Timestamp: " + amazon_timestamp)
    print("Request Timestamp: " + req_timestamp)

    # The scope/action permitted by the signed credentials
    credential_scope = get_credential_scope(req_timestamp, region, service)
    print("Credential Scope: " + credential_scope)

    # Generate and print signed string
    signature_key = get_aws4_signature_key(
        amazon_secret_key, req_timestamp, region, service
    )
    print("Signing Key: " + base64.b64encode(signature_key).decode())
    string_to_sign = get_string_to_sign(amazon_timestamp, credential_scope, user_input)
    print("String to sign: " + json.dumps(string_to_sign))
    signature = hmac.new(
        signature_key, string_to_sign.encode("utf-8"), hashlib.sha256
    ).hexdigest()
    print("Signed String: " + signature)

go run signing_driver.go signing.go $AWS_SECRET_KEY us-west-1 ssm "Hello World!"

// can_req/go/signing.go

ackage main

import (
	hmac "crypto/hmac"
	"crypto/sha256"
	"encoding/json"
	"fmt"
	"os"
	"strings"
	"time"
)

const SIGNING_ALGORITHM = "AWS4-HMAC-SHA256"

func getTimestamps() (string, string) {
	now := time.Now().UTC()
	return now.Format("20060102T150405Z"), now.Format("20060102")
}

func getCredentialScope(request_timestamp string, region string, service string) string {
	return fmt.Sprintf("%s/%s/%s/aws4_request", request_timestamp, region, service)
}

func sign(key string, message string) string {
	mac := hmac.New(sha256.New, []byte(key))
	mac.Write([]byte(message))
	return string(mac.Sum(nil))
}

func signHex(key string, message string) string {
	mac := hmac.New(sha256.New, []byte(key))
	mac.Write([]byte(message))
	return fmt.Sprintf("%x", string(mac.Sum(nil)))
}

func computeSHA256Hash(input string) string {
	hash := sha256.New()
	hash.Write([]byte(input))
	return fmt.Sprintf("%x", string(hash.Sum(nil)))
}

func getStringToSign(amazon_timestamp string, scope string, can_req string) string {
	components := [...]string{SIGNING_ALGORITHM, amazon_timestamp, scope, computeSHA256Hash(can_req)}
	return strings.Join(components[:], "\n")
}

func getAWS4SignatureKey(secret_key string, request_timestamp string, region string, service string) string {
	kdate := sign("AWS4"+secret_key, request_timestamp)
	kregion := sign(kdate, region)
	kservice := sign(kregion, service)
	ksigning := sign(kservice, "aws4_request")
	return ksigning
}

func runDemo() {
	// Get user input from command args
	amazon_secret_key := os.Args[1]
	region := os.Args[2]
	service := os.Args[3]
	user_input := os.Args[4]

	// Fetch the required timestamps
	amazon_timestamp, request_timestamp := getTimestamps()
	fmt.Printf("Amazon Timestamp: %s\n", amazon_timestamp)
	fmt.Printf("Request Timestamp: %s\n", request_timestamp)

	// Get the scope/permitted API action for the signed credentials
	credential_scope := getCredentialScope(request_timestamp, region, service)
	fmt.Printf("Credential Scope: %s\n", credential_scope)

	// Generate and print signed string
	signature_key := getAWS4SignatureKey(amazon_secret_key, request_timestamp, region, service)
	fmt.Printf("Signing Key: %x\n", signature_key)
	string_to_sign := getStringToSign(amazon_timestamp, credential_scope, user_input)
	string_to_sign_formatted, _ := json.Marshal(string_to_sign)
	fmt.Printf("String to sign: `%s`\n", string_to_sign_formatted)
	signature := signHex(signature_key, string_to_sign)
	fmt.Printf("Signed String: " + signature)
}

// can_req/go/signing_driver.go

package main

func main() {
	runDemo()
}

Output

Amazon Timestamp: 20230625T174754Z
Requset Timestamp: 20230625
Credential Scope: 20230625/us-west-1/ssm/aws4_request
Signing Key: 843b458b4664ec9c54e42274a490b2c7cb2802cc104dcba2ad2df8fe71c008ff
String to sign: "AWS4-HMAC-SHA256\n20230625T174754Z\n20230625/us-west-1/ssm/aws4_request\n7f83b1657ff1fc53b92dc18148a1d65dfc2d4b1fa3d677284addd200126d9069"
Signed String: cc1a8368f317707c89b33e8f627f722819ed4d28341fef7b56720103b5d3fe79

Requests

The below steps provide a broad overview for how to successfully perform a canonical request to AWS services.

Canonical Requests

1. Procure AWS credentials, an access key ID and a secret key value, for generating the canonical request. The access key ID will indicate the identity of the caller and the secret value will be used to generate the request signature.

2. Creating the credential scope: This value restricts the request to the target service and region and is of the following format: TIMESTAMP/REGION/SERVICE/SIGNING_VERSION where the timestamp value is of form YYYYMMDD.

3. Generate a SHA256 hash of the parameters included in the request body.

   Pseudocode

   payloadHash = SHA256(CANONICAL_REQUEST_STRING)
   

4. Generate a string containing a list of request headers, with each key/value pair separated by newline characters. Ensure that the headers are listed alphabetically and that the resulting string is also newline terminated.

   Pseudocode

   canonicalHeadersString = ""
   headers.sort()
   for headerName, headerValue in headers:
     canonicalHeadersString += headerName + ":" + headerValue + "\n"
   

5. Generate a string containing all the canonical request information, including the HTTP method, request URI, query parameters, the previously generated header string, alphabetically listed headers used in signing process, and the previously generated request payload hash separated by newline characters.

   Pseudocode

   canonicalRequest = httpMethod + "\n" +
     requestUri + "\m" + 
     queryString + "\n" + 
     canonicalHeadersString + "\m"  + 
     SIGNED_HEADERS + "\m" +
     payloadHash
   

6. Following the steps outline in the previous section, generate the request signature.

7. Generate the authorization header from the following components.

   Pseudocode

   authorizationHeader = `${SIGNING_ALGORITHM} Credential=${amazonKeyId}/${scope}, SignedHeaders=${SIGNED_HEADERS}, Signature=${signature}`;
   

8. Perform an HTTP request with the canonical request parameters.

   Pseudocode

   http_request(endpoint=URI + QUERY_PARAMATERS, headers={header_1=value_1, header_2=value_2, ..., header_n=value_n, Authorization=authorizationHeader}, data=apiParameters, method=httpMethod)
   

Demo

The following are concrete code examples demonstrating how to generate and send a canonical request to list the AWS account users. For those interested, the AWS SDKs can provide further concrete examples for handling canonical requests in a more general manner.

ts-node request.ts $AWS_ACCESS_KEY_ID $AWS_SECRET_KEY

// can_req/ts/request.ts

import * as signing from "./signing";
import * as https from "https";

const METHOD = "GET";
const SIGNING_ALGORITHM = "AWS4-HMAC-SHA256";
const CONTENT_TYPE = "application/x-amz-json-1.1";
const SERVICE = "iam";
const HOST = "iam.amazonaws.com";
const REGION = "us-east-1";
const SIGNED_HEADERS = "content-type;host;x-amz-date";
const CANONICAL_URI = "/";
const CANONICAL_QUERY_STRING = "Action=ListUsers&Version=2010-05-08";

function getCanonicalHeaders(amzTimestamp: string) {
  return [
    "content-type:" + CONTENT_TYPE,
    "host:" + HOST,
    "x-amz-date:" + amzTimestamp + "\n",
  ].join("\n");
}

function getCanonicalRequest(canonicalHeaders: string, payloadHash: string) {
  return [
    METHOD,
    CANONICAL_URI,
    CANONICAL_QUERY_STRING,
    canonicalHeaders,
    SIGNED_HEADERS,
    payloadHash,
  ].join("\n");
}

function getAuthorizationHeader(
  scope: string,
  signature: string,
  amazonKeyId: string
) {
  return `${SIGNING_ALGORITHM} Credential=${amazonKeyId}/${scope}, SignedHeaders=${SIGNED_HEADERS}, Signature=${signature}`;
}

if (require.main === module) {
  // Get user input
  const amazonKeyId = process.argv[2];
  const secretKey = process.argv[3];

  // Get the required timestamp strings
  let [amzTimestamp, reqTimestamp] = signing.getTimestamps();
  console.log("Amazon Timestamp: " + amzTimestamp);
  console.log("Req Timestamp: " + reqTimestamp);

  // Get the scope of the request (the timestamp and the target service)
  const scope = signing.getCredentialScope(reqTimestamp, REGION, SERVICE);
  console.log("Credential Scope: " + scope);
  
  // API  parameters should be listed here when applicable.
  const requestParamters = ``;
  const payloadHash = signing.computeSHA256SignatureHash(requestParamters);

  const headers = getCanonicalHeaders(amzTimestamp);
  const canonicalRequest = getCanonicalRequest(headers, payloadHash);

  //  Get the AWS v4 signing key
  const key = signing.getAWS4SignatureKey(
    secretKey,
    reqTimestamp,
    REGION,
    SERVICE
  );
  const stringToSign = signing.getStringToSign(
    amzTimestamp,
    scope,
    canonicalRequest
  );

  // Sign and output user string
  const signature = signing.signHex(key, Buffer.from(stringToSign));
  console.log("Signature: " + signature);

  const authHeader = getAuthorizationHeader(scope, signature, amazonKeyId);
  console.log("Auth Header: " + authHeader);

  const canReqHeaders = {
    "Accept-Encoding": "identity",
    "Content-Type": CONTENT_TYPE,
    "X-Amz-Date": amzTimestamp,
    Authorization: authHeader,
    "Content-Length": requestParamters.length,
  };

  var options = {
    hostname: HOST,
    path: "/?Action=ListUsers&Version=2010-05-08",
    port: 443,
    method: METHOD,
    headers: canReqHeaders,
  };
  var req = https.request(options, function (res) {
    res.on("data", (d) => {
      process.stdout.write(d);
    });
  });
  req.write(requestParamters);
  req.end();
}

// can_req/ts/signing.ts

import * as crypto from "crypto";

const SIGNING_ALGORITHM = "AWS4-HMAC-SHA256";

export function getTimestamps(): [string, string] {
  const now = new Date();
  const year = now.getUTCFullYear();
  const month = String(now.getUTCMonth() + 1).padStart(2, "0");
  const day = String(now.getUTCDate()).padStart(2, "0");
  const hours = String(now.getUTCHours()).padStart(2, "0");
  const minutes = String(now.getUTCMinutes()).padStart(2, "0");
  const seconds = String(now.getUTCSeconds()).padStart(2, "0");

  const amzTimestamp = `${year}${month}${day}T${hours}${minutes}${seconds}Z`;
  const reqTimestamp = `${year}${month}${day}`;
  return [amzTimestamp, reqTimestamp];
}

export function getCredentialScope(
  reqTimestamp: string,
  region: string,
  service: string
): string {
  return `${reqTimestamp}/${region}/${service}/aws4_request`;
}

export function getStringToSign(
  amzTimestamp: string,
  scope: string,
  message: string
): string {
  return [
    SIGNING_ALGORITHM,
    amzTimestamp,
    scope,
    computeSHA256SignatureHash(message),
  ].join("\n");
}

export function sign(key: Buffer, message: Buffer): Buffer {
  return crypto.createHmac("SHA256", key).update(message).digest();
}

export function signHex(key: Buffer, message: Buffer): string {
  return crypto.createHmac("SHA256", key).update(message).digest("hex");
}

export function computeSHA256SignatureHash(input: string): string {
  return crypto.createHash("SHA256").update(input).digest("hex");
}

export function getAWS4SignatureKey(
  key: string,
  reqTimestamp: string,
  region: string,
  service: string
): Buffer {
  const kDate = sign(Buffer.from("AWS4" + key), Buffer.from(reqTimestamp));
  const kRegion = sign(kDate, Buffer.from(region));
  const kService = sign(kRegion, Buffer.from(service));
  const kSigning = sign(kService, Buffer.from("aws4_request"));
  return kSigning;
}

if (require.main === module) {
  // Get user input
  const secretKey = process.argv[2];
  const region = process.argv[3];
  const service = process.argv[4];
  const userInput = process.argv[5];

  // Get the required timestamp strings
  let [amzTimestamp, reqTimestamp] = getTimestamps();
  console.log("Amazon Timestamp: " + amzTimestamp);
  console.log("Requset Timestamp: " + reqTimestamp);

  // Get the scope of the request (the timestamp and the target service)
  const scope = getCredentialScope(reqTimestamp, region, service);
  console.log("Credential Scope: " + scope);

  //  Get the AWS v4 signing key
  const key = getAWS4SignatureKey(secretKey, reqTimestamp, region, service);
  console.log("Signing Key: " + key.toString("hex"));

  // Prepare string value to sign from user input
  const stringToSign = getStringToSign(amzTimestamp, scope, userInput);
  console.log("String to sign: " + JSON.stringify(stringToSign));

  // Sign and output user string
  const signature = signHex(key, Buffer.from(stringToSign));
  console.log("Signed String: " + signature);
}

node request.js $AWS_ACCESS_KEY_ID $AWS_SECRET_KEY

// can_req/js/request.js

const signing = require(`./signing.js`);
const https = require("https");

const METHOD = "GET";
const SIGNING_ALGORITHM = "AWS4-HMAC-SHA256";
const CONTENT_TYPE = "application/x-amz-json-1.1";
const SERVICE = "iam";
const HOST = "iam.amazonaws.com";
const REGION = "us-east-1";
const SIGNED_HEADERS = "content-type;host;x-amz-date";
const CANONICAL_URI = "/";
const CANONICAL_QUERY_STRING = "Action=ListUsers&Version=2010-05-08";

function getCanonicalHeaders(amzTimestamp) {
  return [
    "content-type:" + CONTENT_TYPE,
    "host:" + HOST,
    "x-amz-date:" + amzTimestamp + "\n",
  ].join("\n");
}

function getCanonicalRequest(canonicalHeaders, payloadHash) {
  return [
    METHOD,
    CANONICAL_URI,
    CANONICAL_QUERY_STRING,
    canonicalHeaders,
    SIGNED_HEADERS,
    payloadHash,
  ].join("\n");
}

function getAuthorizationHeader(scope, signature, amazonKeyId) {
  return `${SIGNING_ALGORITHM} Credential=${amazonKeyId}/${scope}, SignedHeaders=${SIGNED_HEADERS}, Signature=${signature}`;
}

if (require.main === module) {
  // Get user input
  const amazonKeyId = process.argv[2];
  const secretKey = process.argv[3];

  // Get the required timestamp strings
  [amzTimestamp, reqTimestamp] = signing.getTimestamps();
  console.log("Amazon Timestamp: " + amzTimestamp);
  console.log("Request Timestamp: " + reqTimestamp);

  // Get the scope of the request (the timestamp and the target service)
  const scope = signing.getCredentialScope(reqTimestamp, REGION, SERVICE);
  console.log("Credential Scope: " + scope);

  const requestParamters = ``;
  const payloadHash = signing.computeSHA256SignatureHash(requestParamters);

  const headers = getCanonicalHeaders(amzTimestamp);
  const canonicalRequest = getCanonicalRequest(headers, payloadHash);

  //  Get the AWS v4 signing key
  const key = signing.getAWS4SignatureKey(
    secretKey,
    reqTimestamp,
    REGION,
    SERVICE
  );
  console.log("Signing Key: " + key.toString("hex"));

  // Prepare string value to sign from user input
  const stringToSign = signing.getStringToSign(
    amzTimestamp,
    scope,
    canonicalRequest
  );

  // Sign and output user string
  const signature = signing.signHex(key, stringToSign);
  console.log("Signature: " + signature);

  const authHeader = getAuthorizationHeader(scope, signature, amazonKeyId);
  console.log("Auth Header: " + authHeader);

  const canReqHeaders = {
    "Accept-Encoding": "identity",
    "Content-Type": CONTENT_TYPE,
    "X-Amz-Date": amzTimestamp,
    Authorization: authHeader,
    "Content-Length": requestParamters.length,
  };

  var options = {
    hostname: HOST,
    path: "/?" + CANONICAL_QUERY_STRING,
    port: 443,
    method: METHOD,
    headers: canReqHeaders,
  };
  var req = https.request(options, function (res) {
    res.on("data", (d) => {
      process.stdout.write(d);
    });
  });
  req.write(requestParamters);
  req.end(options.body || "").on("error", (err) => {
    console.log("Error: " + err.message);
  });
}

// can_req/js/signing.js

var crypto = require("crypto");

const SIGNING_ALGORITHM = "AWS4-HMAC-SHA256";

function getTimestamps() {
  const now = new Date();
  const year = now.getUTCFullYear();
  const month = String(now.getUTCMonth() + 1).padStart(2, "0");
  const day = String(now.getUTCDate()).padStart(2, "0");
  const hours = String(now.getUTCHours()).padStart(2, "0");
  const minutes = String(now.getUTCMinutes()).padStart(2, "0");
  const seconds = String(now.getUTCSeconds()).padStart(2, "0");

  const amzTimestamp = `${year}${month}${day}T${hours}${minutes}${seconds}Z`;
  const reqTimestamp = `${year}${month}${day}`;
  return [amzTimestamp, reqTimestamp];
}

function getCredentialScope(reqTimestamp, region, service) {
  return `${reqTimestamp}/${region}/${service}/aws4_request`;
}

function getStringToSign(amzTimestamp, scope, message) {
  return [
    SIGNING_ALGORITHM,
    amzTimestamp,
    scope,
    computeSHA256SignatureHash(message),
  ].join("\n");
}

function sign(key, msg) {
  return crypto
    .createHmac("SHA256", key)
    .update(Buffer.from(msg, "utf-8"))
    .digest();
}

function signHex(key, msg) {
  return crypto.createHmac("SHA256", key).update(msg).digest("hex");
}

function computeSHA256SignatureHash(input) {
  return crypto
    .createHash("SHA256")
    .update(Buffer.from(input, "utf-8"))
    .digest("hex");
}

function getAWS4SignatureKey(key, reqTimestamp, region, service) {
  const kDate = sign(Buffer.from("AWS4" + key, "utf-8"), reqTimestamp);
  const kRegion = sign(kDate, region);
  const kService = sign(kRegion, service);
  const kSigning = sign(kService, "aws4_request");
  return kSigning;
}

if (require.main === module) {
  // Get user input
  const secretKey = process.argv[2];
  const region = process.argv[3];
  const service = process.argv[4];
  const userInput = process.argv[5];

  // Get the required timestamp strings
  [amzTimestamp, reqTimestamp] = getTimestamps();
  console.log("Amazon Timestamp: " + amzTimestamp);
  console.log("Request Timestamp: " + reqTimestamp);

  // Get the scope of the request (the timestamp and the target service)
  const scope = getCredentialScope(reqTimestamp, region, service);
  console.log("Credential Scope: " + scope);

  //  Get the AWS v4 signing key
  const key = getAWS4SignatureKey(secretKey, reqTimestamp, region, service);
  console.log("Signing Key: " + key.toString("hex"));

  // Prepare string value to sign from user input
  const stringToSign = getStringToSign(amzTimestamp, scope, userInput);
  console.log("String to sign: " + JSON.stringify(stringToSign));

  // Sign and output user string
  const signature = signHex(key, stringToSign);
  console.log("Signed String: " + signature);
}

module.exports = {
  getTimestamps,
  getCredentialScope,
  computeSHA256SignatureHash,
  getAWS4SignatureKey,
  getStringToSign,
  signHex,
};

python3 request.py $AWS_ACCESS_KEY_ID $AWS_SECRET_KEY

# can_req/py/request.py

from signing import *

import requests
import xml.dom.minidom

# Default API parameters provided here for convenience
METHOD = "GET"
SIGNING_ALGORITHM = "AWS4-HMAC-SHA256"
CONTENT_TYPE = "application/x-amz-json-1.1"
SERVICE = "iam"
HOST = "iam.amazonaws.com"
REGION = "us-east-1"
SIGNED_HEADERS = "content-type;host;x-amz-date"
CANONICAL_URI = "/"
CANONICAL_QUERY_STRING = "Action=ListUsers&Version=2010-05-08"


def get_canonical_headers(amzn_date: str) -> str:
    """Get canonical headers in proper format"""
    return "\n".join(
        [
            "content-type:{}".format(CONTENT_TYPE),
            "host:{}".format(HOST),
            "x-amz-date:{}\n".format(amzn_date),
        ]
    )


def get_canonical_requests(canonical_headers: str, payload_hash: str) -> str:
    """Generate canonical request from the provided headers and payload hash"""
    return "\n".join(
        [
            METHOD,
            CANONICAL_URI,
            CANONICAL_QUERY_STRING,
            canonical_headers,
            SIGNED_HEADERS,
            payload_hash,
        ]
    )


def get_authorization_header(scope: str, signature: str, amazon_key_id: str) -> str:
    """Get the authorization header used for verifying the authenticity of the request"""
    return "{} Credential={}/{}, SignedHeaders={}, Signature={}".format(
        SIGNING_ALGORITHM, amazon_key_id, scope, SIGNED_HEADERS, signature
    )


if __name__ == "__main__":
    # Get provided AWS access credentials
    amazon_key_id = sys.argv[1]
    amazon_secret_key = sys.argv[2]

    endpoint = "https://{}/".format(HOST)
    amazon_timestamp, req_timestamp = get_timestamps()
    print("Amazon Timestamp: " + amazon_timestamp)
    print("Request Timestamp: " + req_timestamp)

    credential_scope = get_credential_scope(req_timestamp, REGION, SERVICE)
    print("Credential Scope: " + credential_scope)

    # API parameters should be listed here when applicable.
    request_paramters = ""
    payload_hash = compute_sha256_hash(request_paramters)

    headers = get_canonical_headers(amazon_timestamp)
    canoniocal_request = get_canonical_requests(headers, str(payload_hash))

    signature_key = get_aws4_signature_key(
        amazon_secret_key, req_timestamp, REGION, SERVICE
    )
    print("Signing Key: " + base64.b64encode(signature_key).decode())
    string_to_sign = get_string_to_sign(
        amazon_timestamp, credential_scope, canoniocal_request
    )
    signature = hmac.new(
        signature_key, (string_to_sign).encode("utf-8"), hashlib.sha256
    ).hexdigest()
    print("Signature: " + signature)

    auth_header = get_authorization_header(credential_scope, signature, amazon_key_id)
    print("Auth Header: " + auth_header)

    # Perform AWS API call
    headers = {
        "Accept-Encoding": "identity",
        "Content-Type": CONTENT_TYPE,
        "X-Amz-Date": amazon_timestamp,
        "Authorization": auth_header,
    }
    resp = requests.get(
        "https://iam.amazonaws.com/?Action=ListUsers&Version=2010-05-08",
        headers=headers,
    )
    print(
        "\nAPI Response:\n"
        + xml.dom.minidom.parseString(resp.content).toprettyxml(indent="", newl="")
    )

# can_req/py/signing.py

import base64, datetime, hashlib, hmac, json, sys

SIGNING_ALGORITHM = "AWS4-HMAC-SHA256"


def get_timestamps() -> tuple[str, str]:
    """Get strings of required timestamps for canonical requests"""
    now = datetime.datetime.utcnow()
    amazon_timestamp = now.strftime("%Y%m%dT%H%M%SZ")
    req_timestamp = now.strftime("%Y%m%d")
    return amazon_timestamp, req_timestamp


def get_credential_scope(req_timestamp: str, region: str, service: str) -> str:
    """Define the scope of the request, which includes the target region and service"""
    return "{}/{}/{}/aws4_request".format(req_timestamp, region, service)


def sign(key: str, msg: str) -> bytes:
    """Generate the HMAC-SHA256 hash of a target string using the provided secret key"""
    return hmac.new(key, msg.encode("utf-8"), hashlib.sha256).digest()


def compute_sha256_hash(input: str) -> str:
    """Create SHA256 hash of a target string"""
    m = hashlib.sha256()
    m.update(input.encode("utf-8"))
    result = m.hexdigest()
    return result


def get_string_to_sign(amzn_date_stamp: str, scope: str, can_req: str) -> str:
    """Get string to sign from request parameters"""
    return "\n".join(
        [SIGNING_ALGORITHM, amzn_date_stamp, scope, compute_sha256_hash(can_req)]
    )


def get_aws4_signature_key(
    key: str, datestamp: str, region: str, service_name: str
) -> bytes:
    """Generature canonical requests signature"""
    kdate = sign(("AWS4" + key).encode("utf-8"), datestamp)
    kregion = sign(kdate, region)
    kservice = sign(kregion, service_name)
    ksigning = sign(kservice, "aws4_request")
    return ksigning


if __name__ == "__main__":
    # Get user input from command args
    amazon_secret_key = sys.argv[1]
    region = sys.argv[2]
    service = sys.argv[3]
    user_input = sys.argv[4]

    # Fetch the required timestamps
    amazon_timestamp, req_timestamp = get_timestamps()
    print("Amazon Timestamp: " + amazon_timestamp)
    print("Request Timestamp: " + req_timestamp)

    # The scope/action permitted by the signed credentials
    credential_scope = get_credential_scope(req_timestamp, region, service)
    print("Credential Scope: " + credential_scope)

    # Generate and print signed string
    signature_key = get_aws4_signature_key(
        amazon_secret_key, req_timestamp, region, service
    )
    print("Signing Key: " + base64.b64encode(signature_key).decode())
    string_to_sign = get_string_to_sign(amazon_timestamp, credential_scope, user_input)
    print("String to sign: " + json.dumps(string_to_sign))
    signature = hmac.new(
        signature_key, string_to_sign.encode("utf-8"), hashlib.sha256
    ).hexdigest()
    print("Signed String: " + signature)

go run request.go signing.go $AWS_ACCESS_KEY_ID $AWS_SECRET_KEY

// can_req/go/request.go

package main

import (
	"bytes"
	"fmt"
	"io/ioutil"
	"net/http"
	"os"
	"strings"
)

const METHOD = "GET"
const CONTENT_TYPE = "application/x-amz-json-1.1"
const SERVICE = "iam"
const HOST = "iam.amazonaws.com"
const REGION = "us-east-1"
const SIGNED_HEADERS = "content-type;host;x-amz-date"
const CANONICAL_URI = "/"
const CANONICAL_QUERY_STRING = "Action=ListUsers&Version=2010-05-08"

func getCanonicalHeaders(amazonDate string) string {
	headers := [...]string{"content-type:" + CONTENT_TYPE, "host:" + HOST, "x-amz-date:" + amazonDate + "\n"}
	return strings.Join(headers[:], "\n")
}

func getCanonicalRequest(canonicalHeaders string, payloadHash string) string {
	requestComponents := [...]string{METHOD, CANONICAL_URI, CANONICAL_QUERY_STRING, canonicalHeaders, SIGNED_HEADERS, payloadHash}
	return strings.Join(requestComponents[:], "\n")
}

func getAuthorizationHeader(scope string, signature string, amazonKeyId string) string {
	return fmt.Sprintf("%s Credential=%s/%s, SignedHeaders=%s, Signature=%s", SIGNING_ALGORITHM, amazonKeyId, scope, SIGNED_HEADERS, signature)
}

func main() {
	// Get user input from command args
	amazon_key_id := os.Args[1]
	amazon_secret_key := os.Args[2]

	// Fetch the required timestamps
	amazon_timestamp, request_timestamp := getTimestamps()
	fmt.Printf("Amazon Timestamp: %s\n", amazon_timestamp)
	fmt.Printf("Request Timestamp: %s\n", request_timestamp)

	// Get the scope of the request (the timestamp and the target service)
	scope := getCredentialScope(request_timestamp, REGION, SERVICE)
	fmt.Printf("Credential Scope: %s\n", scope)

	// API parameters should be listed here when applicable.
	requestParamters := ``
	payloadHash := computeSHA256Hash(requestParamters)

	headers := getCanonicalHeaders(amazon_timestamp)
	canonical_request := getCanonicalRequest(headers, payloadHash)

	signature_key := getAWS4SignatureKey(amazon_secret_key, request_timestamp, REGION, SERVICE)
	fmt.Printf("Signing Key: %x\n", signature_key)
	string_to_sign := getStringToSign(amazon_timestamp, scope, canonical_request)

	// Sign and output user string
	signature := signHex(signature_key, string_to_sign)
	fmt.Printf("Signature: %s\n", signature)

	auth_header := getAuthorizationHeader(scope, signature, amazon_key_id)
	fmt.Printf("Auth Header: %s\n", auth_header)

	client := &http.Client{}
	endpoint := fmt.Sprintf("https://%s/?"+CANONICAL_QUERY_STRING, HOST)
	req, err := http.NewRequest(METHOD, endpoint, bytes.NewBuffer([]byte(requestParamters)))
	if err != nil {
		fmt.Println(err)
		return
	}
	req.Header.Add("Accept-Encoding", "identity")
	req.Header.Add("Content-Type", CONTENT_TYPE)
	req.Header.Add("X-Amz-Date", amazon_timestamp)
	req.Header.Add("Authorization", auth_header)
	resp, err := client.Do(req)
	if err != nil {
		fmt.Println(err)
		return
	}
	defer resp.Body.Close()

	body, err := ioutil.ReadAll(resp.Body)
	if err != nil {
		fmt.Println(err)
		return
	}

	fmt.Println(string(body))
}

// can_req/go/signing.go

ackage main

import (
	hmac "crypto/hmac"
	"crypto/sha256"
	"encoding/json"
	"fmt"
	"os"
	"strings"
	"time"
)

const SIGNING_ALGORITHM = "AWS4-HMAC-SHA256"

func getTimestamps() (string, string) {
	now := time.Now().UTC()
	return now.Format("20060102T150405Z"), now.Format("20060102")
}

func getCredentialScope(request_timestamp string, region string, service string) string {
	return fmt.Sprintf("%s/%s/%s/aws4_request", request_timestamp, region, service)
}

func sign(key string, message string) string {
	mac := hmac.New(sha256.New, []byte(key))
	mac.Write([]byte(message))
	return string(mac.Sum(nil))
}

func signHex(key string, message string) string {
	mac := hmac.New(sha256.New, []byte(key))
	mac.Write([]byte(message))
	return fmt.Sprintf("%x", string(mac.Sum(nil)))
}

func computeSHA256Hash(input string) string {
	hash := sha256.New()
	hash.Write([]byte(input))
	return fmt.Sprintf("%x", string(hash.Sum(nil)))
}

func getStringToSign(amazon_timestamp string, scope string, can_req string) string {
	components := [...]string{SIGNING_ALGORITHM, amazon_timestamp, scope, computeSHA256Hash(can_req)}
	return strings.Join(components[:], "\n")
}

func getAWS4SignatureKey(secret_key string, request_timestamp string, region string, service string) string {
	kdate := sign("AWS4"+secret_key, request_timestamp)
	kregion := sign(kdate, region)
	kservice := sign(kregion, service)
	ksigning := sign(kservice, "aws4_request")
	return ksigning
}

func runDemo() {
	// Get user input from command args
	amazon_secret_key := os.Args[1]
	region := os.Args[2]
	service := os.Args[3]
	user_input := os.Args[4]

	// Fetch the required timestamps
	amazon_timestamp, request_timestamp := getTimestamps()
	fmt.Printf("Amazon Timestamp: %s\n", amazon_timestamp)
	fmt.Printf("Request Timestamp: %s\n", request_timestamp)

	// Get the scope/permitted API action for the signed credentials
	credential_scope := getCredentialScope(request_timestamp, region, service)
	fmt.Printf("Credential Scope: %s\n", credential_scope)

	// Generate and print signed string
	signature_key := getAWS4SignatureKey(amazon_secret_key, request_timestamp, region, service)
	fmt.Printf("Signing Key: %x\n", signature_key)
	string_to_sign := getStringToSign(amazon_timestamp, credential_scope, user_input)
	string_to_sign_formatted, _ := json.Marshal(string_to_sign)
	fmt.Printf("String to sign: `%s`\n", string_to_sign_formatted)
	signature := signHex(signature_key, string_to_sign)
	fmt.Printf("Signed String: " + signature)
}

Output

Amazon Timestamp: 20230625T182331Z
Req Timestamp: 20230625
Credential Scope: 20230625/us-west-2/ssm/aws4_request
Signature: 5a5ebdb8797f0568c19f2ea45d70bc77309ae11eff9a69990202871db3cdbcde
Auth Header: AWS4-HMAC-SHA256 Credential=BKDZY123A2Z4DV37XABC/20230625/us-west-2/ssm/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-target, Signature=5a5ebdb8797f0568c19f2ea45d70bc77309ae11eff9a69990202871db3cdbcde

API Response:
<?xml version="1.0" ?>
<ListUsersResponse xmlns="https://iam.amazonaws.com/doc/2010-05-08/">
  <ListUsersResult>
    <IsTruncated>false</IsTruncated>
    <Users>
      <member>
        <Path>/</Path>
        <UserName>username</UserName>
        <Arn>arn:aws:iam::563175520123456789104152:user/username</Arn>
        <UserId>BIDAFHH7AT44NDI6LEIYP</UserId>
        <CreateDate>2019-07-10T23:13:11Z</CreateDate>
      </member>
    </Users>
  </ListUsersResult>
  <ResponseMetadata>
    <RequestId>482ac647-51b2-47cd-9aa2-2eafe6bb8c8b</RequestId>
  </ResponseMetadata>
</ListUsersResponse>

Review

Session Tokens

While IAM users may use their access credentials to programmatically access AWS services and resources, it can be desirable to leverage short-term credentials in cases where users may want to provide temporary access or perform a privileged action in a limited timeframe. Secure Token Service, abbreviated STS, can be used both for obtaining short-lived AWS tokens and obtaining client information. Such credentials consist of the following properties:¹

• Access Key ID: Unique identifier for provided credentials
• Secret Access Key: Used for signing API requests
• Session Token: Used to authenticate the API request
• Expiration Time: Date when credentials are no longer valid

aws sts get-session-token

{
    "Credentials": {
        "AccessKeyId": "ASIAYBJ8AT44NA7VMLB1",
        "SecretAccessKey": "0aZJykOJAUQV3lalUAgoXcNoXZ2OKfxau//tAck+",
        "SessionToken": "IQoJb3JpZ2lu ... paRFtuA5mBac=",
        "Expiration": "2023-09-19T08:42:24+00:00"
    }
}

Global and Regional Endpoints

STS is a global service, meaning the API is available in all AWS regions. The default global endpoint, https://sts.amazonaws.com, is hosted in us-east-1. However, AWS recommends leveraging region-specific API endpoints when possible for the following reasons:

• Lower STS response latency
• Adds redundancy should certain STS endpoints become available
• Increase token validity

For example, while it is possible to request eu-central-1-scoped credentials from the global STS endpoint, services hosted in eu-central-1 should target https://sts.eu-central-1.amazonaws.com for decreased latency. The following two STS commands were executed on an AWS server instance hosted in eu-central-1 to further illustrate this:

$ export TIMEFORMAT=%R
$ time aws sts get-session-token  --region eu-central-1
...
0.834
$ time aws sts get-session-token  --region eu-central-1 --endpoint-url https://sts.eu-central-1.amazonaws.com
...
0.469

Repeated calls from the eu-central-1 server demonstrate this latency decrease consistently:

SDK and CLI Integration

While cloud developers may manually fetch STS credentials to sign and perform AWS API calls, the provided CLI and SDK tools automatically manage calls to STS in the background. Developers should only consider using the STS endpoints and direct API calls to AWS should only be used when developing in languages that are not supported by AWS or when fine-grain control over AWS API calls is required. See here for a list of supported languages. (https://aws.amazon.com/developer/tools/)

The AWS CLI and SDK will typically look for credentials in the following order:

• Credentials provided to the SDK or CLI invocation
• From available environment variables (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY)
• The .aws/config and .aws/credentials folder: AWS will use the credentials found in the credentials file. Similarly, it will request the STS role for a given profile.

The easiest way to use STS credentials with CLI calls is simply to set the environment variables as such:

$ export AWS_ACCESS_KEY_ID=”STS_PROVIDED_ACCESS_KEY_ID”
$ export AWS_SECRET_ACCESS_KEY=”STS_PROVIDED_SECRET_ACCESS_KEY”
$ aws sts get-caller-identity

Caller Information

In addition to fetching short term credentials, STS can be used to acquire information pertaining to the active caller or given access key.

Caller Identity

To fetch information pertaining to the credentials actively used in signing requests, STS provides GetCallerIdentityInfo endpoint. This endpoint returns the following information:¹

• UserId: The account unique ID of the caller associated with the credentials.
• Account: The globally unique 12-digit ID of the parent account
• Arn: The access resource number for the resource

aws sts get-caller-identity

{
    "UserId": "AIDAYJG7SY44NDI7LOIES",
    "Account": "012345678910",
    "Arn": "arn:aws:iam::012345678910:user/username"
}

Access Key

For a given access key ID, it is possible to get the parent account ID of the credentials using the GetAccessKeyInfo endpoint.²

aws sts get-access-key-info --access-key-id $AWS_ACCESS_KEY_ID

{
    "Account": "012345678910"
}

Review

Introduction

IAM, which stands for Identity Access Management, is an AWS service used to manage API and web console access to cloud resources.

Overview

When a user attempts to access an AWS resource through the API or web console, an HTTP request is sent to the endpoint for the target AWS service. After verifying the request signature, it is evaluated against all relevant IAM access policies, which dictate the allowed actions for the caller. If there exists at least one rule granting access and no explicit deny rule in any policies or permission boundaries, then the request is granted. Otherwise, it is rejected.

Features

IAM primarily handles all aspects of authentication and authorization when accessing cloud services.¹ Authentication determines the validity of the identity of a user while authorization determines whether a given identity is permitted to perform a given action.

Identity

Identity is an entity that enables access to an AWS account’s cloud resources.¹ IAM provides the following entities for managing identity information:

• Users: Represents an entity, typically a human user, that accesses AWS resources or services
• Groups: Manages access for a collection of AWS users.
• Roles: Temporarily assumed by an identity to perform a given set of actions.

Access

Once the identity of the caller has been established, the next step is to determine whether they are authorized to perform the requested action. To accomplish this, IAM evaluates the identity against all relevant policies, which specifies the allowed and restricted actions for a given user. These IAM policies can be directly assigned to users or groups or through Attribute Access Control (ABAC), which associates a policy with an identity based on the identity’s tags.²

Management

In addition to providing services to define identity information and access, IAM also provides services to ensure best practices are enforced regarding access. Some of this tooling includes analyzing user’s access patterns to refine permissions, identifying unused permissions, and other auditing capabilities.

Non-Features

IAM handles authentication and authorization when accessing resources via the AWS APIs. It is not intended for the following use-cases:

• Network Security: IAM does not evaluate non-AWS API traffic to AWS resources, such as network connections to public servers or databases.
• Application Identity Management: IAM is not intended to provide identity functionality for third-party applications and services, such as storing user credentials, and handling of login/account-reset flows.