Manage data products

Create a new data product

1.10.10 2.0.4

To create a new data product:

Java Python Kotlin Raw REST API

FluentSearch assets = Table.select() // (1)
    .where(Table.CERTIFICATE_STATUS.eq(CertificateStatus.VERIFIED))
    .where(Table.ATLAN_TAGS.eq("Marketing"))
    .build();
DataProduct dp = DataProduct.creator("Marketing Influence", // (2)
        DataDomain.refByQualifiedName("default/domain/marketing"), // (3)
        assets) // (4)
    .build(); // (5)
AssetMutationResponse response = dp.save(); // (6)

1. When defining a data product, you must define the assets within it. These are defined through a search, so that the assets included can be automatically managed. In this example, we are selecting all verified tables that have a tag of Marketing.
2. You must provide a human-readable name for your data product.
3. You must also provide the domain in which the data product should exist.
4. And finally the search that was defined earlier, to define which assets to include in the data product.
5. You then need to build the object.
6. And then you can save() the object you've built to create the new data product in Atlan.

from pyatlan.client.atlan import AtlanClient
from pyatlan.model.assets import DataProduct, Table
from pyatlan.model.fluent_search import CompoundQuery, FluentSearch
from pyatlan.model.enums import CertificateStatus

client = AtlanClient()

assets = (
  FluentSearch()
  .where(CompoundQuery.active_assets())
  .where(CompoundQuery.asset_type(Table))
  .where(Table.CERTIFICATE_STATUS.eq(CertificateStatus.VERIFIED.value))
  .where(Table.ATLAN_TAGS.eq("Marketing"))
).to_request()  # (1)

product = DataProduct.create(
  name="Marketing Influence",  # (2)
  asset_selection=assets,  # (3)
  domain_qualified_name="default/domain/marketing"  # (4)
)

response = client.asset.save(product)  # (5)

1. When defining a data product, you must define the assets within it. These are defined through a search, so that the assets included can be automatically managed. In this example, we are selecting all verified tables that have a tag of Marketing.
2. You must provide a human-readable name for your data product.
3. You must provide the index search request that was defined earlier, to define which assets to include in the data product.
4. You must also provide the domain in which the data product should exist.
5. And then you can save() the object you've built to create the new data product in Atlan.

val assets = Table.select() // (1)
    .where(Table.CERTIFICATE_STATUS.eq(CertificateStatus.VERIFIED))
    .where(Table.ATLAN_TAGS.eq("Marketing"))
    .build()
val dp = DataProduct.creator("Marketing Influence",  // (2)
        DataDomain.refByQualifiedName("default/domain/marketing"),  // (3)
        assets) // (4)
    .build() // (5)
val response = dp.save() // (6)

1. When defining a data product, you must define the assets within it. These are defined through a search, so that the assets included can be automatically managed. In this example, we are selecting all verified tables that have a tag of Marketing.
2. You must provide a human-readable name for your data product.
3. You must also provide the domain in which the data product should exist.
4. And finally the search that was defined earlier, to define which assets to include in the data product.
5. You then need to build the object.
6. And then you can save() the object you've built to create the new data product in Atlan.

 {
   "entities": [
     {
       "typeName": "DataProduct", // (1)
       "attributes": {
         "qualifiedName": "default/domain/marketing/product/marketingInfluence", // (2)
         "name": "Marketing Influence", // (3)
         "parentDomainQualifiedName": "default/domain/marketing", // (4)
         "superDomainQualifiedName": "default/domain/marketing", // (5)
         "dataProductAssetsDSL": "{\"query\": {\"attributes\": [\"__traitNames\", \"connectorName\", \"__customAttributes\", \"certificateStatus\", \"tenantId\", \"anchor\", \"parentQualifiedName\", \"Query.parentQualifiedName\", \"AtlasGlossaryTerm.anchor\", \"databaseName\", \"schemaName\", \"parent\", \"connectionQualifiedName\", \"collectionQualifiedName\", \"announcementMessage\", \"announcementTitle\", \"announcementType\", \"announcementUpdatedAt\", \"announcementUpdatedBy\", \"allowQuery\", \"allowQueryPreview\", \"adminGroups\", \"adminRoles\", \"adminUsers\", \"category\", \"credentialStrategy\", \"connectionSSOCredentialGuid\", \"certificateStatus\", \"certificateUpdatedAt\", \"certificateUpdatedBy\", \"classifications\", \"connectionId\", \"connectionQualifiedName\", \"connectorName\", \"dataType\", \"defaultDatabaseQualifiedName\", \"defaultSchemaQualifiedName\", \"description\", \"displayName\", \"links\", \"link\", \"meanings\", \"name\", \"ownerGroups\", \"ownerUsers\", \"qualifiedName\", \"typeName\", \"userDescription\", \"displayDescription\", \"subDataType\", \"rowLimit\", \"queryTimeout\", \"previewCredentialStrategy\", \"policyStrategy\", \"policyStrategyForSamplePreview\", \"useObjectStorage\", \"objectStorageUploadThreshold\", \"outputPortDataProducts\"], \"dsl\": {\"from\": 0, \"query\": {\"bool\": {\"filter\": {\"bool\": {\"filter\": [{\"term\": {\"__state\": {\"value\": \"ACTIVE\", \"case_insensitive\": false}}}, {\"term\": {\"__typeName.keyword\": {\"value\": \"Table\", \"case_insensitive\": false}}}, {\"term\": {\"certificateStatus\": {\"value\": \"VERIFIED\", \"case_insensitive\": false}}}, {\"term\": {\"__traitNames\": {\"value\": \"Marketing\", \"case_insensitive\": false}}}]}}}}}, \"suppressLogs\": true}, \"filterScrubbed\": true}",
         // (6)
         "dataProductAssetsPlaybookFilter": "{\"condition\":\"AND\",\"isGroupLocked\":false,\"rules\":[]}" 
         // (7)
       },
       "relationshipAttributes": {
         "dataDomain": {
           "typeName": "DataDomain",
           "uniqueAttributes": { // (8)
             "qualifiedName": "default/domain/marketing"
           }
         }
       }
     }
   ]
}

1. The typeName must be exactly DataProduct.
2. Provide a human-readable name for your data product.
3. Ensure the qualifiedName follows the pattern: <parentDomainQualifiedName>/product/<lowerCamelCaseName>.
4. Provide a parentDomainQualifiedName for the data domain under which you want to create this product.
5. Provide a superDomainQualifiedName for the data domain under which you want to create this product. If creating a product under sub-domains, this should be the qualified name of the root-level domain.

6. Provide the DSL that defines the assets to include in the data product as an embedded JSON string.

   Use SDK to create data products

   The above data products assets DSL requires a filter as a nested object construct within an outer bool, rather than a list or array. It's recommended to create data products via SDK as it handles this complexity automatically.

7. Specify the default playbook filter to define which assets are shown in the data product UI.

8. Specify the qualifiedName of the data domain under which you want to create this product.

Retrieve a data product

1.8.4 1.8.1

To retrieve a data product, use its qualified name:

Java Python Kotlin Raw REST API

DataProduct dp = DataProduct.get("default/product/marketingInfluence"); // (1)

1. If no exception is thrown, the returned object will be a non-null data domain.

from pyatlan.client.atlan import AtlanClient
from pyatlan.model.assets import DataProduct

client = AtlanClient()

product = client.asset.get_by_qualified_name(
    asset_type=DataProduct,  # (1)
    qualified_name="default/product/marketingInfluence"
)

1. If no exception is thrown, the returned object will be non-null and of the type requested.

val dp = DataProduct.get("default/product/marketingInfluence") // (1)

1. If no exception is thrown, the returned object will be a non-null data domain.

// (1)

1. In the case of retrieving an asset, all necessary information is included in the URL of the request. There is no payload for the body of the request.

Update a data product

1.8.4 1.8.1

To update a data product:

Java Python Kotlin Raw REST API

DataProduct dp = DataProduct.updater("default/product/marketingInfluence", // (1)
        "Marketing Influence")
    .userDescription("Now with a description!") // (2)
    .build(); // (3)
AssetMutationResponse response = dp.save(); // (4)

1. Use the updater() method to update a data product, providing the qualifiedName and name of the data product.
2. You can chain onto the updater any other enrichment, for example changing the product's description.
3. You then need to build the object.
4. You can then save() the object you've built to update the data product in Atlan.

from pyatlan.client.atlan import AtlanClient
from pyatlan.model.assets import DataProduct

client = AtlanClient()

product = DataProduct.create_for_modification( # (1)
    qualified_name="default/product/marketingInfluence", # (2)
    name="Marketing Influence", # (3)
)
product.user_description = "Now with a description!" # (4)

response = client.asset.save(product) # (5)

1. Use the create_for_modification() method to update a data product.
2. You must provide the qualifiedName of the data product.
3. You must provide the name of the data product.
4. You can then add on any other updates, such as changing the user description of the data product.
5. To update the data product in Atlan, call the save() method with the object you've built.

val dp = DataProduct.updater("default/product/marketingInfluence", // (1)
        "Marketing Influence")
    .userDescription("Now with a description!") // (2)
    .build() // (3)
val response = dp.save() // (4)

1. Use the updater() method to update a data product, providing the qualifiedName and name of the data product.
2. You can chain onto the updater any other enrichment, for example changing the product's description.
3. You then need to build the object.
4. You can then save() the object you've built to update the data product in Atlan.

{
  "entities": [
    {
      "typeName": "DataProduct", // (1)
      "attributes": {
        "name": "Marketing Influence", // (2)
        "qualifiedName": "default/product/marketingInfluence", // (3)
        "userDescription": "Now with a description!" // (4)
      }
    }
  ]
}

1. The typeName must be exactly DataProduct.
2. Human-readable name for your data product.
3. You must provide the the qualifiedName of the data product to update.
4. You can add on any other updates, such as changing the user description of the data product.

Delete a data product

1.8.4 1.8.1

Soft-delete (archive)

To soft-delete, or archive, a data product:

Java Python Kotlin Raw REST API

AssetDeletionResponse response = DataProduct.delete("218c8144-dc39-43a5-b0c0-9eeb4d11e74a"); // (1)

1. To archive a data product in Atlan, call the DataProduct.delete() method with the GUID of the data product.

from pyatlan.client.atlan import AtlanClient

client = AtlanClient()

client.asset.delete_by_guid("218c8144-dc39-43a5-b0c0-9eeb4d11e74a") # (1)

1. To archive a data product in Atlan, call the asset.delete_by_guid() method with the GUID of the data product.

val response = DataProduct.delete("218c8144-dc39-43a5-b0c0-9eeb4d11e74a") // (1)

1. To archive a data product in Atlan, call the DataProduct.delete() method with the GUID of the data product.

// (1)

1. All the details for deleting the data product are specified in the URL directly. Note that you must provide the GUID of the data product to delete it.

Hard-delete (purge)

To permanently delete (purge) a data product:

Java Python Kotlin Raw REST API

AssetDeletionResponse response = DataProduct.purge("218c8144-dc39-43a5-b0c0-9eeb4d11e74a"); // (1)

1. To permanently delete a data product in Atlan, call the DataProduct.purge() method with the GUID of the data product.

from pyatlan.client.atlan import AtlanClient

client = AtlanClient()

client.asset.purge_by_guid("218c8144-dc39-43a5-b0c0-9eeb4d11e74a") # (1)

1. To permanently delete a data product in Atlan, call the asset.purge_by_guid() method with the GUID of the data product.

val response = DataProduct.purge("218c8144-dc39-43a5-b0c0-9eeb4d11e74a") // (1)

1. To permanently delete a data product in Atlan, call the DataProduct.purge() method with the GUID of the data product.

// (1)

1. All the details for deleting the data product are specified in the URL directly. Note that you must provide the GUID of the data product to delete it.

---