Azure Function Activity - Azure Data Factory & Azure Synapse (2024)

  • Article

APPLIES TO: Azure Function Activity - Azure Data Factory & Azure Synapse (1)Azure Data Factory Azure Function Activity - Azure Data Factory & Azure Synapse (2)Azure Synapse Analytics

Tip

Try out Data Factory in Microsoft Fabric, an all-in-one analytics solution for enterprises. Microsoft Fabric covers everything from data movement to data science, real-time analytics, business intelligence, and reporting. Learn how to start a new trial for free!

The Azure Function activity allows you to run Azure Functions in an Azure Data Factory or Synapse pipeline. To run an Azure Function, you must create a linked service connection. Then you can use the linked service with an activity that specifies the Azure Function that you plan to execute.

Create an Azure Function activity with UI

To use an Azure Function activity in a pipeline, complete the following steps:

  1. Expand the Azure Function section of the pipeline Activities pane, and drag an Azure Function activity to the pipeline canvas.

  2. Select the new Azure Function activity on the canvas if it is not already selected, and its Settings tab, to edit its details.

    Azure Function Activity - Azure Data Factory & Azure Synapse (3)

  3. If you do not already have an Azure Function linked service defined, select New to create a new one. In the new Azure Function linked service pane, choose your existing Azure Function App url and provide a Function Key.

    Azure Function Activity - Azure Data Factory & Azure Synapse (4)

  4. After selecting the Azure Function linked service, provide the function name and other details to complete the configuration.

Azure Function linked service

The return type of the Azure function has to be a valid JObject. (Keep in mind that JArray is not a JObject.) Any return type other than JObject fails and raises the user error Response Content is not a valid JObject.

Function Key provides secure access to function name with each one having separate unique keys or master key within a function app. Managed identity provides secure access to the entire function app. User needs to provide key to access function name. For more information, see the function documentation for more details about Function access key

PropertyDescriptionRequired
TypeThe type property must be set to: AzureFunctionYes
Function app urlURL for the Azure Function App. Format is https://<accountname>.azurewebsites.net. This URL is the value under URL section when viewing your Function App in the Azure portalYes
Function keyAccess key for the Azure Function. Click on the Manage section for the respective function, and copy either the Function Key or the Host key. Find out more here: Work with access keysYes
AuthenticationThe authentication method used for calling the Azure Function. The supported values are 'System-assigned managed identity' or 'anonymous'.Yes
Resource IDThe App (client) ID of the Azure Function. Switch to Authentication section for the respective function, and get the App (client) ID under Identity provider. This property will be displayed when you use system-assigned managed identity. For more information, see Configure your App Service or Azure Functions app to use Microsoft Entra login.No

Note

When you use anonymous authentication, ensure that you have taken down your identity on the Azure Function side.

Azure Function activity

PropertyDescriptionAllowed valuesRequired
NameName of the activity in the pipelineStringYes
TypeType of activity is ‘AzureFunctionActivity’StringYes
Linked serviceThe Azure Function linked service for the corresponding Azure Function AppLinked service referenceYes
Function nameName of the function in the Azure Function App that this activity callsStringYes
MethodREST API method for the function callString Supported Types: "GET", "POST", "PUT"Yes
HeaderHeaders that are sent to the request. For example, to set the language and type on a request: "headers": { "Accept-Language": "en-us", "Content-Type": "application/json" }String (or expression with resultType of string)No
BodyBody that is sent along with the request to the function api methodString (or expression with resultType of string) or object.Required for PUT/POST methods

See the schema of the request payload in Request payload schema section.

Routing and queries

The Azure Function Activity supports routing. For example, if your Azure Function has the endpoint https://functionAPP.azurewebsites.net/api/<functionName>/<value>?code=<secret>, then the functionName to use in the Azure Function Activity is <functionName>/<value>. You can parameterize this function to provide the desired functionName at runtime.

Note

The functionName for Durable Functions should be taken from the route property of the function's binding in its JSON definition, to include its routing information. Simply using the functionName without the route detail included will result in a failure because the Function App cannot be found.

The Azure Function Activity also supports queries. A query must be included as part of the functionName. For example, when the function name is HttpTriggerCSharp and the query that you want to include is name=hello, then you can construct the functionName in the Azure Function Activity as HttpTriggerCSharp?name=hello. This function can be parameterized so the value can be determined at runtime.

Timeout and long-running functions

Azure Functions times out after 230 seconds regardless of the functionTimeout setting you've configured in the settings. For more information, see this article. To work around this behavior, follow an async pattern or use Durable Functions. The benefit of Durable Functions is that they offer their own state-tracking mechanism, so you don't need to implement your own state-tracking.

Learn more about Durable Functions in this article. You can set up an Azure Function Activity to call the Durable Function, which will return a response with a different URI, such as this example. Because statusQueryGetUri returns HTTP Status 202 while the function is running, you can poll the status of the function by using a Web Activity. Set up a Web Activity with the url field set to @activity('<AzureFunctionActivityName>').output.statusQueryGetUri. When the Durable Function completes, the output of the function will be the output of the Web Activity.

Sample

You can find a sample that uses an Azure Function to extract the content of a tar file here.

Related content

Learn more about supported activities in Pipelines and activities.

Azure Function Activity - Azure Data Factory & Azure Synapse (2024)

FAQs

What is the difference between Azure functions and synapse? ›

Azure Functions is a popular tool to create REST APIs to expose services, both internally and externally. Synapse workspace is an example where APIs from other teams can be leveraged. Data engineers can use Synapse pipelines to ingest metadata, send notifications and/or run small computations exposed by other teams.

Is Azure Synapse same as Azure Data Factory? ›

Azure Data Factory is a cloud-based data integration service that allows you to create, schedule, and orchestrate data pipelines. Azure Synapse Analytics, on the other hand, is a cloud-based big data and data warehousing service that brings together big data and data warehousing.

What is Azure Function activity from synapse pipeline? ›

The Azure Function activity allows you to run Azure Functions in an Azure Data Factory or Synapse pipeline. To run an Azure Function, you must create a linked service connection. Then you can use the linked service with an activity that specifies the Azure Function that you plan to execute.

What is the difference between Azure Data Factory and Azure Function? ›

Azure Data Factory (ADF) and Function Apps are pivotal components in Azure's data ecosystem. ADF is a cloud-based data integration service, while Function Apps provide serverless compute resources for executing code snippets.

What is the purpose of Azure Synapse? ›

Azure Synapse Analytics is an enterprise analytics service that accelerates time to insight across data warehouses and big data systems. It brings together the best of SQL technologies used in enterprise data warehousing, Apache Spark technologies for big data, and Azure Data Explorer for log and time series analytics.

How many types of Azure Functions are there? ›

There are currently four durable function types in Azure Functions: activity, orchestrator, entity, and client. The rest of this section goes into more details about the types of functions involved in an orchestration.

What is the old name of Azure Synapse? ›

Azure Synapse Analytics is an analytics service that brings together enterprise data warehousing and Big Data analytics. Dedicated SQL pool (formerly SQL DW) refers to the enterprise data warehousing features that are available in Azure Synapse Analytics.

Is Azure Synapse an ETL tool? ›

Azure Synapse and Snowflake offer excellent ETL (extract, transform, load) tools. They are both powerful and feature-rich, making them excellent choices for businesses of all sizes. However, there are some critical differences between the two platforms that should be considered when choosing an ETL tool.

What is the alternative of Azure Synapse? ›

Top Microsoft Azure Synapse Analytics Competitors

The top alternative solutions include Azure Data Factory, Snowflake, and KNIME. The alternatives are sorted based on how often peers compare the solutions.

What are the types of activities in Azure Data Factory? ›

The two most important components you will come across are pipelines and activities. A pipeline acts as a logical collection of activities working together for task completion. The three types of activities you can run with Microsoft Azure Data Factory include data movement, data transformation, and control activities.

What is the difference between Azure Data Factory and Azure pipelines? ›

While Azure Data Factory is a data integration and orchestration service, Azure Pipelines focuses on continuous integration and continuous delivery (CI/CD) of applications.

How do you execute pipeline activity with Azure Synapse? ›

Search for pipeline in the pipeline Activities pane, and drag an Execute Pipeline activity to the pipeline canvas. Select the new Execute Pipeline activity on the canvas if it is not already selected, and its Settings tab, to edit its details. Select an existing pipeline or create a new one using the New button.

What is the function of Azure Data Factory? ›

Integrated Security: Azure Data Factory offers integrated security features such as Entra ID integration and role-based access control to control access to dataflows. This feature increases security in data processing and protects your data.

Why should I use Azure Data Factory? ›

In Azure Data Factory, monitor all your activity runs visually and improve operational productivity by setting up alerts proactively to monitor your pipelines. These alerts can then appear within Azure alert groups, ensuring that you're notified in time to prevent downstream or upstream problems before they happen.

What is the difference between Azure function and Azure function app? ›

Azure App Service is a powerful web application hosting platform. Azure Functions, built on top of the App Service infrastructure, enables you to easily build serverless and event-driven compute workloads. Both services are frequently used in multitenant solutions.

What is the difference between Azure database and Azure Synapse? ›

SQL DB offers easy maintenance and a predictable cost structure. For a more complex data warehouse, Synapse provides a one-stop shop for service integrations allowing it to effectively manage massive amounts of data for a more complicated data warehouse.

What is the difference between Azure Functions and Azure function app? ›

In summary, Azure Web Apps are ideal for traditional web hosting scenarios, providing greater control over resources and tailored scaling options, while Azure Functions are designed for serverless, event-driven computing scenarios, offering dynamic scaling based on workload requirements.

What is the difference between Azure DevOps and Azure Synapse? ›

Azure Synapse Analytics is an analytics service that brings together enterprise data warehousing and Big Data analytics. Formerly known as Azure SQL Data Warehouse. Azure DevOps provides build and release services to support continuous integration and delivery of solutions.

Top Articles
Fintech Review 2023: Predictions and Outlook for Germany, Austria, and Switzerland
Why, How, & Where To Invest Your Emergency Fund To Beat Inflation
$4,500,000 - 645 Matanzas CT, Fort Myers Beach, FL, 33931, William Raveis Real Estate, Mortgage, and Insurance
Fort Morgan Hometown Takeover Map
NYT Mini Crossword today: puzzle answers for Tuesday, September 17 | Digital Trends
Breaded Mushrooms
Prosper TX Visitors Guide - Dallas Fort Worth Guide
Terraria Enchanting
Obituary (Binghamton Press & Sun-Bulletin): Tully Area Historical Society
Achivr Visb Verizon
People Portal Loma Linda
Animal Eye Clinic Huntersville Nc
2016 Ford Fusion Belt Diagram
iOS 18 Hadir, Tapi Mana Fitur AI Apple?
Wilmot Science Training Program for Deaf High School Students Expands Across the U.S.
065106619
Abortion Bans Have Delayed Emergency Medical Care. In Georgia, Experts Say This Mother’s Death Was Preventable.
Kiddle Encyclopedia
Iu Spring Break 2024
Dover Nh Power Outage
Ge-Tracker Bond
Decosmo Industrial Auctions
Www.craigslist.com Savannah Ga
Optum Urgent Care - Nutley Photos
R. Kelly Net Worth 2024: The King Of R&B's Rise And Fall
Betaalbaar naar The Big Apple: 9 x tips voor New York City
Seeking Arrangements Boston
Wnem Tv5 Obituaries
Obituaries Milwaukee Journal Sentinel
Albert Einstein Sdn 2023
Wood Chipper Rental Menards
Cornedbeefapproved
Craigslist Northern Minnesota
Taktube Irani
Tire Pro Candler
Utexas Baseball Schedule 2023
Manuel Pihakis Obituary
Blackstone Launchpad Ucf
Pill 44615 Orange
How to Destroy Rule 34
Flashscore.com Live Football Scores Livescore
Telegram update adds quote formatting and new linking options
Scanning the Airwaves
Instafeet Login
Lyca Shop Near Me
Vision Source: Premier Network of Independent Optometrists
M Life Insider
Winta Zesu Net Worth
Oklahoma City Farm & Garden Craigslist
Aloha Kitchen Florence Menu
Kenwood M-918DAB-H Heim-Audio-Mikrosystem DAB, DAB+, FM 10 W Bluetooth von expert Technomarkt
Hughie Francis Foley – Marinermath
Latest Posts
Article information

Author: Otha Schamberger

Last Updated:

Views: 6191

Rating: 4.4 / 5 (55 voted)

Reviews: 86% of readers found this page helpful

Author information

Name: Otha Schamberger

Birthday: 1999-08-15

Address: Suite 490 606 Hammes Ferry, Carterhaven, IL 62290

Phone: +8557035444877

Job: Forward IT Agent

Hobby: Fishing, Flying, Jewelry making, Digital arts, Sand art, Parkour, tabletop games

Introduction: My name is Otha Schamberger, I am a vast, good, healthy, cheerful, energetic, gorgeous, magnificent person who loves writing and wants to share my knowledge and understanding with you.