Provides conversion from athena outputs to strongly defined data models.
Getting started
Given the following data struct you define:
type MyModel struct {
ID int `athenaconv:"id"`
Name string `athenaconv:"name"`
SourceComputersCount int64 `athenaconv:"source_computers_count"`
SourceComputerExternalIDs []string `athenaconv:"source_computer_ids"`
SourceComputerNames []string `athenaconv:"source_computer_names"`
TestTimestamp time.Time `athenaconv:"test_timestamp"`
TestDate time.Time `athenaconv:"test_date"`
TestBool bool `athenaconv:"test_bool"`
}
And the following sql:
select
id,
name,
count(source_id) as source_computers_count,
array_agg(source_id) as source_computer_ids,
array_agg(source_name) as source_computer_names,
timestamp '2012-10-31 08:11:22' as test_timestamp,
date '2021-12-31' as test_date,
true as test_bool
from my_glue_catalog_table
group by id, name
You can convert your athena.GetQueryResultOutput object to strongly-typed struct MyModel by doing this:
mapper, err := athenaconv.NewMapperFor(reflect.TypeOf(MyModel{}))
if err != nil {
handleError(err)
}
var mapped []interface{}
mapped, err = mapper.FromAthenaResultSetV2(ctx, queryResultOutput.ResultSet)
if err != nil {
handleError(err)
}
for _, mappedItem := range mapped {
mappedItemModel := mappedItem.(*MyModel)
fmt.Printf("%+v\n", *mappedItemModel)
}
Supported data types
See conversion.go in this repo and supported data types in athena for more details.
| Athena data type | Go data type | Comments |
|---|---|---|
| varchar | string | |
| boolean | bool | |
| integer | int/int32 | |
| bigint | int64 | |
| timestamp | time.Time | |
| date | time.Time | |
| array | []string | Individual items within array should not contain comma |
| other data types | string | Other data types currently unsupported, default to string (no conversion) |
Supported AWS SDK version
Roadmap / items to review
- Add more data type support in conversion.go
- Review usage of logging (best practice for logging in golang packages)
