S3 assets

Operations on S3 assets (buckets, objects).

S3 assets for now can only be created and managed via these APIs: they are not crawled separately through any of our connectors.

In general, these should be:

  • Created in top-down order (connection, then bucket, then object)
  • Deleted in bottom-up order (objects, then buckets, then connections)

(Although if you want to delete everything in a connection your better avenue is the packaged connection delete utility in the UI.)

The creation order is important as each contained object will refer to its parent. The parent object must therefore exist before attempting to create any children. However, note that each request takes an array of entities. Therefore, the contained objects can still be created in bulk once the parent object exists.

(For example: you can create a bucket in one API call, and then in a second API call you could create hundreds of objects within that bucket. These hundreds of objects can all be created in a single API call rather than hundreds of separate API calls.)

Once again, you can find step-by-step walkthroughs and explanations of the creation process and models in our Developer Portal for both AWS S3 and Google Cloud Storage.