# Downloading AWS CodeWatch Log

We run our build with AWS codebuild and our repository is in Bitbucket.

We have mono repo and the build we run inside code-build which is triggered on every PR creation.

Since we do a lot of stuff in build including container creation, and automated tests, it is sometimes required to analyze the entire log of one particular build.

AWS store the log in a JSON format which looks like this

```json
{
'events': [
            'timestamp': 123,
            'message': 'string',
            'ingestionTime': 123
    ],
    'nextForwardToken': 'string',
    'nextBackwardToken': 'string'
 }
```

When you go to Cloud watch and try to analyze the log , you can not see the entire log. You can either search from the events or keep clicking load more button.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1686195159606/b03433a2-479f-48a1-b8e6-57c6bfb3abee.png align="center")

When you click on ***View entire log***, it takes you to cloud watch and you still have to keep clicking ***Load more.***

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1686195302965/6b76c801-5b5b-4f17-954c-effbad2a5a51.png align="center")

This is very inconvenient and sometime we would like to look at the entire log ( just the message ) and do a search like grep locally.

I looked at a tool called awslogs which is hosted in GitHub here [https://github.com/jorgebastida/awslogs](https://github.com/jorgebastida/awslogs)

It is an awesome tool, it allows me to search, and filter streams but doesn't let me have an entire log of a single stream.

At this point, I decided to write a small Python program using boto3 to get this job done for me.

The code is hosted here

[https://github.com/vgaur/simple-aws-log-extract](https://github.com/vgaur/simple-aws-log-extract)

It takes three parameter

* group -> The AWS log group name
    
* stream -> The AWS log stream inside the group
    
* outfile -> The local file where the content will be downloaded.
    

```bash
python pull-aws-log.py --group XXX --stream YYY --outfile ZZZ
```

This does the job but its not handy.

So I updated script to delete and create the outfile. \[ In future I can just take a location and create file based on stream name \]

Next I updated my zshrc file to make this a function

```bash
pullawslog() {
   python3 /Users/XXX/tools/awslog/pull-aws-log.py "$@"
}
```

I also updated my script to hide group name since I was only dealing with one group.

Now I can call the script like this

```bash
pullawslog --stream xx-ee-ss
```

Just one command and you can download entire stream log locally.