Task Factory users running version 2020.1.4 or older (released prior to May 27, 2020): There's an important Task Factory update. Please visit here for more details.
Connection Manager
Google Cloud OAuth2 Connection Manager
This connection manager is used with the Google Cloud Source and Google Cloud Rest Destination. Before connecting, users must obtain a Client ID, Client Secret, a Storage Project Id, and a Redirect URL from Google's APIs & Services Credentials dashboard.
To connect to Google Cloud Rest, complete the following:
| Option | Description |
|---|---|
| Client Id | The Client ID is found in Google's APIs & Services - Credentials console. |
| Client Secret | The Client Secret is found in Google's APIs & Services - Credentials console. |
| Access Token | Select Get Token to open the Token Getter window. |
| Storage Project Id | Enter the storage Id for your container after you retrieve your access token. To find this number, select this link and then select the Gear icon on the left side (Settings.) The Storage Project Id is found in the section Identifying your project - x-goog-project-id: 123456789 (In this example, the Storage Project Id is 123456789.) |
Token Getter
| Option | Description |
|---|---|
| Redirect Uri | The redirect Uri configured in Google's APIs & Services - Credentials console. |
| Client Id | This field should be auto-completed from the previous window with your Client ID. |
| Client Secret | This field should be auto-completed from the previous window with your Client Secret. |
| Scopes | Selects which permission to be used in the call:
Additional Information: For more information about Scopes, see Google's Cloud Storage Authentication. |
| Get Access Token button | Opens a mini browser within the window to login to your Google account and grant access. |
Google Drive Oauth2 Connection Manager
Used with the Google Drive Source and Google Drive Destination. Before connecting, users must obtain a Client ID, Client Secret, and Redirect URL from Google's APIs & Services Credentials dashboard.
Connection Settings
To connect to Google Drive, complete the following:
| Option | Description |
|---|---|
| Client Id | The Client ID is found in Google's APIs & Services - Credentials console. |
| Client Secret | The Client Secret is found in Google's APIs & Services - Credentials console. |
| Access Token | Select Get Token to open the Token Getter window. |
| Use Team Drive | Select this option if your application uses a Team Drive to share documents with other users. |
Token Getter
| Option | Description |
|---|---|
| Redirect Uri | The redirect Uri configured in Google's APIs & Services - Credentials console. |
| Client Id | This field should be auto-completed from the previous window with your Client ID. |
| Client Secret | This field should be auto-completed from the previous window with your Client Secret. |
| Scopes | Selects which permission to be used in the call:
Additional Information: For more information about Scopes, see Google's Cloud Storage Authentication. |
| Get Access Token button | Opens a mini browser within the window to login to your Google account and grant access. |
The Token Getter window automatically closes and the rest of the connection manager automatically completes. A small browser window opens at the bottom of the Token Getter Window. Log in to your Google Drive application's account and select Allow to grant access.
Proxy Configuration
Select the Proxy tab to connect to your Google Drive via a proxy connection.
| Option | Description |
|---|---|
| Proxy Host | Identifies the proxy address. |
| Proxy Port | Identifies the port used by the proxy. |
| Proxy User Name | Enter the user name for proxies that require user authentication. |
| Proxy Password | Enter the password for proxies that require user authentication. |
Source
Google Cloud Rest Source and Google Drive Rest Source
Source
Begin by creating a connection manager that connects to the service's storage container.
Format
The following formats are available:
Delimited Format
Users can manually configure the delimiter , quote identifiers, the lines to skip, and lines to scan. Select Detect Formats to configure these fields automatically.
| Option | Description |
|---|---|
| Delimiter | Identifies the character that separates columns. The default value is the comma (,). |
| Quote | Identifies the character used to indicate a value (if present). |
| Lines To Skip | Users can designate the number of lines to skip at the beginning of the delimited file. |
| Auto | Automatically detects which lines to skip. |
| Lines To Scan | Determines the number of lines to scan when detecting the file. |
| Detect Formats | Select this button to begin scanning the chosen file. |
Detected Delimited View - After the format is detected, a new view appears that allows users to configure or change different properties of the file.
Json Format
| Option | Description |
|---|---|
| Root Json Path | JSON queries can return multiple levels, therefore, this field Identifies the root to be used. |
| Output Columns | In this window, users can add and remove columns, define their name, token path (defined within the raw data), data type, length, precision, scale, and code page. |
| Date Parse Handling | Specifies how date formatted strings are parsed when reading JSON text.
|
| Date Time Zone Handling | Specifies how to treat time values converted between string and DateTime.
|
| Float Parse Handling | Specifies how floating point numbers are parsed.
|
XML Format
| Option | Description |
|---|---|
| Root XPath Query | XML queries can return multiple levels, therefore, this field Identifies the root to be used. |
| Namespaces: |
|
| Columns | Users can create, remove, and configure the name, index (zero-based), data type, length, precision, and scale of the columns being extracted from the file. |
Output Columns
Users can select which columns to include/exclude in the output. Additionally, users can rename the columns by selecting into its corresponding output name.
Error Handling
| Option | Description |
|---|---|
| Not Used | Error handling is not used and no error constraint is available. |
| Ignore Failure | All errors are ignored and the package continues to execute. |
| Redirect Row | All error rows are directed to an error output. |
| Fail Component | (Default selection) On error, the component fails and the package execution stops. |
Preview
Select Preview Data after configuring the previous tabs to view an output sample.
Destination
Google Cloud Destination and Google Drive Destination
Target
Begin by creating a connection manager that connects to a Google Storage container. After a connection manager is created, the source window populates with files and folders. Select the desired file to continue configuration.
| Option | Description |
|---|---|
| Target Item Name | Defines the name and extension of the file you would like to create when the component is executed. (For example: MyNewXmlList.xml) |
Delimited Format
| Option | Description |
|---|---|
| Delimiter | Identifies the character that separates text. |
| Quote | Identifies the character (single or double quote) that indicates value. |
| Include Header Row | Writes the column headers at the beginning of the document. |
| Encoding | Selects which encoding is used to create the document. |
| Culture | Used to identify the document's language culture. |
Json Array Format
| Option | Description |
|---|---|
| Generate | Select to automatically generate the Row Body metadata. (Users can also add custom header and footer data.) |
| Encoding | Select which encoding is used to create the document. |
| Culture | Used to identify the document's language culture. |
XML Array Format
| Option | Description |
|---|---|
| Generate | Select to automatically generate the Row Body metadata. Users can also add custom header and footer data. |
| Encoding | Select which encoding is used to create the document. |
| Culture | Used to identify the document's language culture. |
