Skip to content
  • Blog
  • Events
  • Help
  • Careers
  • Contact
New Signature
  • About
      • Company

        Cognizant Microsoft Business Group is dedicated to changing the way businesses innovate, transform and run based on a unique cloud operating model. You will now be redirected to our new microsite to learn more.

        View Company

      • Awards

        As a company, we are regularly recognized within the IT industry as well as the communities we serve.

        View All Awards
      • News

        Read the most up-to-date corporate announcements, Microsoft technology updates, innovative business solutions and learn more about how the Cognizant Microsoft Business Group can take your business even farther.

        View News

      • Partners

        New Signature works with a number of outstanding technology companies to deliver the best experiences to our customers.

        View Partners
      • Leadership

        Cognizant Microsoft Business Group’s executive team is comprised of innovative leaders with proven experience and deep industry expertise. You will now be redirected to our new microsite to learn more.

        View Leadership

      • Industries

        Our solutions are tailored to empower organizations across a wide range
        of industries.

        View Industry Experience
    Close
  • Solutions
      • Intelligent Enterprise
        Solutions

        Going Digital
        Unleash cloud capability, deliver change and compete at speed with a Microsoft digital operating model, enabling you to work more efficiently as you transform your IT environment. Learn More

      • Featured Solution

        Secure Cloud
        In a world of constant threat, ensuring that your underlying cloud platform is protected is the first step on your organization’s journey towards a secure, compliant operating environment. Learn More
      • Intelligent Workplace
        Solutions

      • Secure Workplace

        Work Anywhere

        Endpoint Health

        Identity Health

        Teamwork Support



        VIEW WORKPLACE SOLUTIONS
      • Intelligent Cloud
        Solutions

      • App Factory

        Azure Accelerator

        Azure Launchpad

        Azure Launchpad for DevOps

        Application Health

        Platform Health

        VIEW ClOUD SOLUTIONS
    Close
  • Services
      • Begin your journey towards becoming a digital business with GO, our unique end-to-end framework based on the Microsoft Cloud Adoption Framework.

        GO DIGITAL OPERATING MODEL
      • Intelligent Enterprise

      • Consulting

        We go beyond just technology to help your organization understand how digital can help you uniquely differentiate and better serve your employees and customers.

        VIEW ENTERPRISE SERVICES
      • Intelligent Workplace

        • Identity

          Identity is your new first-line-of-defense. It’s vital to your users and clients that your identity platform is properly configured and secured.

          Endpoint

          Whether your devices are on-premises or remote, personal or business-owned, we can ensure they are properly managed and protected.

          Teamwork

          Today’s workforce is collaborating than ever before.  We can empower your current teams with tomorrow’s progressive technologies.

          VIEW WORKPLACE SERVICES
        • Intelligent Cloud

          • Platform

            The cloud is no longer some future-state. It’s the here and now. Adopting a cloud-first platform is one of the best ways to maintain a future-proofed competitive advantage.

            Applications

            We build cloud-native apps and modernize legacy systems with the power of Azure to give your organization a competitive edge.

            Data

            We can help your organization create secure, scalable data platforms to deliver simpler and more sophisticated insights to your business.

            VIEW CLOUD SERVICES
        Close
      • Client Stories
          • Case Studies

            Browse a comprehensive list of companies who have created successful partnerships and experienced transformative solutions with New Signature.

            View All Case Studies

          • Featured Case Study TalkTalk Modern Workplace

            New Signature worked with TalkTalk to define a new Modern Workplace solution based on Microsoft 365, which kept the user firmly at the center of the transformation.
            View Case Study

          • Testimonials

            We love transforming our customers businesses, take a look at what they have to say about New Signature.

            View Testimonials

          • Featured Testimonial Davis Construction

            With New Signature’s help, Davis was able to take a progressive step forward by migrating their private branch exchange (PBX) phone system to a Voice of Internet Protocol (VoIP) system.
            View Testimonial

        Close
      • Explore
          • Guides & Ebooks

            Dive deeper into education with your team by leveraging our expert-developed guides and eBooks.

            View All Guides & Ebooks

          • Infographics

            Rich with statistics and information, our infographics are great tools for quick but insightful learning.

            View All Infographics
          • Podcast: Office Explorers

            Join Kat and Rob monthly as they chat with New Signature experts and explore the world of O365.

            Listen to Podcasts

          • Videos

            Visit our videos stream to access recorded webinars, service information and to learn more about us.

            WATCH ALL VIDEOS
          • Flyers

            Searching for information about our services? Our flyers are a great takeaway for all those details.

            VIEW ALL FLYERS

          • Featured Stream

            Learn more about the tooling and expertise required to unlock productivity and mobilize your teams.

            MODERN WORKPLACE
        Close
        Close
      Blog

      Utilizing Azure to Facilitate Near-Real-Time Data Synchronization into Dynamics 365

      New Signature / Blog / Utilizing Azure to Facilitate Near-Real-Time Data Synchronization into Dynamics 365
      November 10, 2020November 10, 2020| Colin McGuire
      • Facebook
      • Twitter
      • LinkedIn
      • Print
      Azure

      The Problem

      On a recent project, we were faced with the challenge of keeping data synchronized between an external data source that was constantly being updated and Dynamics 365, which was serving primarily as the database for an externally facing Dynamics Portal website.

      There were two goals in mind:

      1. Keep the strain on the source system to a minimum;
      2. Data should be synchronized as close to real-time as possible

      The Solution

      The solution we came up with involves three main Azure components:

      1. Queue storage
      2. Logic Apps
      3. Function Apps

      The first step of the process is to get the data we need into Azure. In this case, the source database (Epicor) has the ability to perform custom actions and/or run custom code when a record is modified in any way (created, updated, deleted). Using this functionality, we wrote small pieces of code that identify when a record that we need is updated and pushes the relevant data, as JSON, into an Azure Storage Queue that is specifically for that record-type. The workflow is simple, and the code is very lightweight, minimizing any strain on Epicor itself.

      The next step of the process is to process the data that is now in the Azure Queue. This is done via an Azure Logic App. The Logic App is set up to watch a specific queue. Once a record appears in that queue, the Logic App picks it up for processing. The main step of the processing is to POST the JSON from the queue message to an Azure Function App we’ve created to process the record. If the POST to the Function App returns a 200 OK HTTP response, the record is deleted from the Azure Queue. If any other response is received, the record is put back in the queue to attempt to be processed again.

      The Azure Function App, mentioned as part of the previous step, is the last piece of our data sync puzzle. This is code written to take in JSON and process it as an entity that belongs in Dynamics 365. It can be called like any RESTful web service. It determines based on the data it takes in if the record is new or an update to an existing record, then performs the necessary work to translate the data from JSON into the appropriate entity in Dynamics. If all is successful, it returns a ‘200 OK’ HTTP response. If not, it returns a response relevant to the problem that occurred.

      Azure pieces configuration

      In the short description of the data sync flow above, I’ve only briefly touched on each Azure piece, queues, logic apps and functions, and for the most part they’re all being used in a very simple way. However, it’s worth noting that each have multiple features and, in some cases, can be configured in different ways based on how you want them to act in specific scenarios.

      For example, Azure Queues have a concept of a “Dequeue Count”. This is the number of times the item has been removed from the queue. If this number gets high, it generally indicates that whatever is processing the message is failing with regards to that message repeatedly. Monitoring this value can allow for a proactive response to potential errors.

      For Logic Apps, when processing messages from an Azure Queue, the default configuration is to process up to 25 messages at a time in parallel. In certain scenarios, this may not be desirable functionality. For example, if records needed to be processed in order, you’d want to configure the Logic App to only pick up one message at a time. This can be done by altering the configuration of the step that monitors the queue and setting the concurrency value.

      Summary

      The example laid out above is just one way to solve the problem that was posed to our team. It allowed us to meet our goals of near-real-time data synchronization between the two systems while leaving a minimal impact on the source system. The pieces configured in Azure were easy to set up, allowing us to get the solution in place in a reasonable timeframe, but also powerful enough to handle a large volume of data.

       

      About the Author
      Colin McGuire is currently a Technical Lead at New Signature, focusing on solution design and facilitating development tasks. He enjoys working with his team to see solutions develop from concepts into finished products. When he’s not working, he can often be found playing board games with friends.

      Categories
      Tips and Tricks
      Contact New Signature

      Blog Posts

      • Better Together: Dynamics 365 & Power Platform
      • How to Modernize Your Apps Securely in Azure – Webinar
      • Transforming Your Healthcare with Dynamics 365
      • How Well Do You Know Your Customer?

      Events

      Tue 23

      Cloud Native Transformation: Increase Resilience, Scalability and Application Security with Modern Apps in Azure

      March 23 @ 10:00 am - 11:00 am EDT
      Apr 08

      Power Platform Webinar Series: Build Customer Relationships at Scale

      April 8 @ 10:00 am - 10:45 am EDT

      View More

      New Signature
      New Signature HQ
      901 K Street NW, Suite 450
      Washington, DC 20001
      Phone: 202-452-5923
      New Signature Canada HQ
      7th Floor, 5140 Yonge Street
      Toronto, ON M2N 7J8
      Phone: 416-971-4267
      New Signature UK HQ
      57 Bermondsey Street
      London SE1 3XJ
      Phone: +44 (0) 845-402-1752

      About

      • Company
      • Awards
      • News
      • Leadership
      • Partners
      • Industries

      Solutions

      • Intelligent Enterprise Solutions
      • Intelligent Workplace Solutions
      • Intelligent Cloud Solutions

      Services

      • GO
      • Intelligent Enterprise
      • Intelligent Workplace
      • Intelligent Cloud

      Client Stories

      • Client Stories
      • Testimonials

      Explore

      • Guides & Ebooks
      • Podcasts
      • Flyers
      • Infographics
      • Videos
      Copyright © 2021 New Signature
      • Blog
      • Events
      • Careers
      • Help
      • Anti Slavery
      • Privacy Policy
      • Contact
      • About
        • Company
        • Awards
        • News
        • Leadership
        • Partners
        • Industries
      • Services
        • GO
        • Intelligent Enterprise
        • Intelligent Workplace
        • Intelligent Cloud
      • Client Stories
        • Case Studies
        • Testimonials
      • Technologies
      • Explore
        • Guides & Ebooks
        • Infographics
        • Podcast: Office Explorers
        • Videos
        • Flyers
      • Blog
      • Events
      • Careers
      • Contact
      • Search
      Cookie Settings
      New Signature uses "Required Cookies" to run our website, "Functional Cookies" used by third parties to personalise marketing, including social media features.

      Change your preferences by clicking the “Cookie Settings” link at the bottom of every page. Learn more about cookies in our Cookie Policy and our Privacy Policy. By clicking the “Accept Cookies” button below, you consent to our use of cookies.

      Please note that “Required Cookies” will be set regardless of your consent.
      Cookie SettingsAccept Cookies
      Privacy & Cookies Policy
      Performance

      Performance Cookies provide Content Delivery Network assets that deliver faster site content delivery capabilities.

      Required

      These cookies are required mainly in order to deliver Multilanguage site capabilities.

      Functional

      Functional Cookies allow us to provided advanced media capabilities including videos, surveys and other multimedia capabilities.

      Disabling Functional cookies will block the playing of videos and other multimedia site components.

      Targeting

      Targeting Cookies are used to capture user information in order for New Signature to deliver better user experiences.

      Save & Accept