Task Factory Azure

Note:  Task Factory components can be used with Azure databases. As of the 2018.2.3 release, Task Factory can also be used with Azure Data Factory.

Connection Manager

Note:  Azure Storage connection manager is available for SQL versions 2012 and higher.

Azure Storage Connection Manager

The Azure Storage Connection Manager is used to connect to an Azure Machine Learning blob storage.

FunctionDescription
Storage Account NameThe username used to login to an Azure ML account.
Storage Account KeyThe key generated within the Azure ML account.

Task Factory Azure Storage Connection Manager
Azure Rest Connection Manager

Azure Rest Connection Manager

Connection Properties

Used with the Azure Rest Source and Azure Rest Destination.

Task Factory Azure Rest Connection Manager Connection Properties

OptionDescription
Storage Account Name
Storage account name provided by Azure to connect to the desired container.
Storage Account Key
The account key provided by Azure.
 Endpoint Suffix
Endpoint suffix provided by Azure. Default: core.windows.net

Proxy Configuration

Task Factory Azure Rest Connection Manager Proxy Configuration

OptionDescription
Proxy Host
 Identifies the proxy address.
Proxy Port
Identifies the port used by the proxy.
Proxy User Name
Enter the user name for proxies that require user authentication.
Proxy Password
Enter the password for proxies that require user authentication.
Azure ML Batch Execution Task

Note:  Azure ML Batch Execution is available for SQL versions 2012 and higher.

Azure ML Batch Execution Task

Task IconTask Description
Task Factory Azure ML Batch Execution Task IconThe Azure ML Batch Execution Task is used to execute a batch job for an Azure ML web service. See the Azure Storage Connection Manager to learn more about setting up the connection manager.

Task Factory Azure ML Batch Execution

FunctionDescription
Storage ContainerIdentifies the container within Azure storage where the input and output files are stored.
Input FileThe name of the source file located in the Azure storage container.
Output FileThe name of the destination file located in the Azure storage container.

Note:  This does not need to exist.
Web Service Output NameThe name of the web service output defined in the Azure ML experiment web service.
Base Web Service UrlThe request Url defined in the batch execution api documentation for your web service. To obtain this Url, users need to do the following in Azure ML Studio:

  • Select the web service group.
  • Select the Batch Execution Url in the default endpoint.
  • Select the Request Url until the end of the word jobs and copy.
Execution TimeoutThe amount of time in seconds before the component sends a cancel command.
ParametersIf users have a web service with parameters, this is where to define them.
Azure ML Source

Note:  Azure ML is available for SQL versions 2012 and higher.

Azure ML Source

Task IconTask Description
Task Factory Azure ML Source IconThe Azure ML Storage Source retrieves data from blob storage on Azure for an ML web service. See the Azure Storage Connection Manager to learn more about setting up the connection manager.

Task Factory Azure ML Source

FunctionDescription
Storage ContainerIdentifies the container within Azure storage where the data will be retrieved.
File NameThe name of the .csv file located in the Azure storage container.
Skip First RowThis option is selected by default and used if the first row of data contains headers.
Azure ML Service Metadata URIThe base web service URI with the bolded parameters added to the base Url (e.g.https://ussouthcentral.services.azureml.net/odata/workspaces/3b47fa
82350146aaa16a80d10ed01ba7/services/4171be6bcb584f20afa396ab8590
196c/$metadata.) To obtain the URI, users need to do the following in Azure ML Studio:

  • Select the web service group.
  • Select the Batch Execution URI in the default endpoint. 
  • Select the Request URI up to the word jobs and copy (see screenshot below):
    Task Factory Azure ML Source Batch Execution job example
Refresh MetadataSelecting the button updates the Destination Column window. By default, all columns are included in the output.
Row Error HandlingAdditional Information:  See the Task Factory Error Row Handling article for more information about this functionality.
Azure Rest Source

Azure Rest Source

Source

Task Factory Azure Rest Source

Begin by creating a connection manager that connects to the service's storage container. 

Format

The following formats are available: 

Delimited Format

Users can manually configure the delimiter, quote identifiers, the lines to skip, and lines to scan.  Select Detect Formats to configure these fields automatically. 

Task Factory Azure Rest Source Format Delimited

OptionDescription
DelimiterIdentifies the character that separates columns. The default value is the comma (,).
QuoteIdentifies the character used to indicate a value (if present).
Lines To SkipUsers can designate the number of lines to skip at the beginning of the delimited file.
AutoAutomatically detects which lines to skip.
Lines To ScanDetermines the number of lines to scan when detecting the file.
 Detect FormatsSelect this button to begin scanning the chosen file.

Detected Delimited View - After the format is detected, a new view appears that allows users to configure or change different properties of the file.

Json Format

Task Factory Azure Rest Source Format Json

OptionDescription
 Root Json PathJson queries can return multiple levels, therefore, this field Identifies the root to be used.
Output ColumnsIn this window, users can add and remove columns, define their name, token path (defined within the raw data), data type, length, precision, scale, and code page.  
Date Parse HandlingSpecifies how date formatted strings are parsed when reading JSON text.

  • None - Strings are not parsed as a date type and are read as strings.
  • DateTime - Date formatted strings are parsed as DateTime.
  • DateTime Offset - Date formatted strings are parsed as DateTimeOffset.
Date Time Zone HandlingSpecifies how to treat time values converted between string and DateTime.

  • Local - Treats the time as local (UTC times convert to local times.)
  • Utc - Treats the time as UTC. This converts local time to UTC.
  • Unspecified - Treats the time as local time if a DateTime is being converted to a string. If a string is being converted to DateTime, convert to a local time if a time zone is specified.
  • RoundtripKind - Time zone is preserved when converting.

Float Parse Handling 

Specifies how floating point numbers are parsed.

  • Double - Floating point numbers are parsed as a double datatype.
  • Decimal - Floating point numbers are parsed as a decimal datatype.

XML Format

Task Factory Azure Rest Source Format Xml

OptionDescription
 Root XPath Query Xml queries can return multiple levels, therefore, this field Identifies the root to be used.
Namespaces:
  • Prefix - Identifies the prefix used in an xml namespace. Example: In the element \<SalesDetails xmlns:url="http://www.site.com"> the prefix is url.
  • Uri - The url or address identified in the namespace.

ColumnsUsers can create, remove, and configure the name, index (zero-based), data type, length, precision, and scale of the columns being extracted from the file.

Output Columns

Users can select which columns to include/exclude in the output. Additionally, users can rename the columns by selecting into its corresponding output name.

Task Factory Azure Rest Source Output Columns

Error Handling

Task Factory Azure Rest Source Error Handling

OptionDescription
Not UsedError handling is not used and no error constraint is available.
Ignore FailureAll errors are ignored and the package continues to execute.
Redirect RowAll error rows are directed to an error output.
 Fail Component(Default selection) On error, the component fails and the package execution stops.

Preview

Select Preview Data after configuring the previous tabs to view an output sample.

Task Factory Azure Rest Source Preview

Azure ML Destination

Note:  Azure ML is available for SQL versions 2012 and higher.

Azure ML Destination

Destination IconDestination Description
Task Factory Azure ML Destination IconThe Azure ML Destination is used to send data to a blob storage on Azure storage to be consumed by an Azure ML web service. This is the input file of the Azure ML batch execution task. See the Azure Storage Connection Manager to learn more about setting up the connection manager.

Task Factory Azure ML Destination

FunctionDescription
Storage ContainerIdentifies the container within Azure storage where the data is stored.
File NameThe name of the file in the Azure storage container.
Azure ML Service Metadata UrlThe base web service Url with the bolded parameters added to the base Url (e.g. https://ussouthcentral.services.azureml.net/odata/workspaces
/3b47fa82350146aaa16a80d10ed01ba7/services/4171be6bcb584f20afa
396ab8590196c/$metadata.) To obtain the Url, users need to do the following in Azure ML Studio:

  • Select the web service group.
  • Select the Batch Execution Url in the default endpoint.
  • Select the Request Uri up to the word "jobs" and copy (see screenshot below):


Task Factory Azure ML Destination Batch Execution job example

Output OptionsThis selection includes header columns in the output by default.
 Refresh MetadataSelecting this button updates the Input and Destination columns window (input and destination columns with the same names are auto-mapped.)
Azure Rest Destination

Azure Rest Destination

Target

Begin by creating a connection manager that connects to an Azure Storage container. After a connection manager is created, the source window populates with files and folders. Select the desired file to continue configuration. 

Task Factory Azure Rest Destination Target

OptionDescription
Target Item NameDefines the name and extension of the file you would like to create when the component is executed. (For example: MyNewXmlList.xml)

Delimited Format

Task Factory Azure Rest Destination Format Delimited

OptionDescription
DelimiterIdentifies the character that separates text.
QuoteIdentifies the character (single or double quote) that indicates value.
Include Header RowWrites the column headers at the beginning of the document.
EncodingSelects which encoding is used to create the document.
CultureUsed to identify the document's language culture.

Json Array Format

Task Factory Azure Rest Destination Format JsonArray

OptionDescription
GenerateSelect to automatically generate the Row Body metadata. Users can also add custom header and footer data.
EncodingSelect which encoding is used to create the document.
CultureUsed to identify the document's language culture.

XML Array Format

Task Factory Azure Rest Destination Format XmlArray

OptionDescription
GenerateSelect to automatically generate the Row Body metadata. Users can also add custom header and footer data.
EncodingSelect which encoding is used to create the document.
CultureUsed to identify the document's language culture.