Lambda runner

The lambda runner triggers lambda functions to read / produce to the Kafka clusters.

The lambda function needs to be deployed beforehand using the zoe CLI. The deployment context of the lambda function needs to be configured in the lambda section of the runners config :

runners:
  default: lambda
  config:
    lambda:
      nameSuffix: null  # optional: if not null, this would be appended at the end of the function name
      deploy:
        subnets: ["subnet-xxxxxxx"]
        securityGroups: ["sg-yyyyyyy"]
        memory: 512
        timeout: 500
      credentials:
        type: "profile"
        name: "my-profile"
      awsRegion: eu-west-1

To deploy the lambda function using the configured deployment context:

zoe -v lambda deploy

Zoe deploys the lambda function in 2 steps :

It deploys a minimal set of resources needed by the lambda using CloudFormation (an S3 bucket, AWS roles, etc.). For more details on the resources deployed, checkout the cloudformation template.
It deploys the lambda function itself by uploading the local Zoe JAR into AWS and registering the lambda within AWS. Zoe jar path needs to be set and must point to a valid zoe core jar downloaded beforehand (Todo: a guide for this).

Once the lambda function is deployed, you can use the lambda runner to interact with your cluster:

zoe -c my-cluster -r lambda topics consume input -n 10

Supported credentials

The authentication method is configured in the credentials section of the lambda runner config. 3 Authentication types are supported :

Default: this will use the default AWS credentials chain to authenticate with AWS.
```
credentials:
   type: "default"
```
Profile: uses the configured profile to authenticate with AWS. The chosen profile needs to be configured in ~/.aws/credentials.
```
credentials:
   type: "profile"
   name: "my-profile"
```
Static: uses the statically set keys to authenticate with AWS. The keys themselves can be secrets resolved at runtime by the secrets provider.
```
credentials:
   type: "static"
   accessKey: "accessKey"
   secretAccessKey: "secretAccessKey"
```

How to deal with an unsupported authentication method

There are currently a bunch of other authentication methods that zoe doesn't support yet. Few examples are: MFA in the command line, the new AWS Single Sign On, etc.

If you are dealing with such an unsupported authentication method, you can easily workaround it by generating an STS session and let zoe make use it. To make the process seamless, you can use the awesome AWS Vault that makes it almost transparent.

For zoe to be able to discover STS sessions, you need to use the default credentials type:

runners:
  default: lambda
  config:
    lambda:
      credentials:
        type: "default"
      ...

You can use AWS Vault to make generating STS sessions transparent with zoe by using:

Command

aws-vault exec my-aws-profile-with-sso -- zoe topics list

You can even simplify the process further by creating an alias:

# Create an alias
alias zoe-pro='aws-vault exec my-aws-profile-with-sso -- zoe'

# Use it here
zoe-pro topics list