Overview of Presentation
              Project background and objectives
              Challenges to managing data from emerging technologies
              What is big data?
              Why should agencies move toward the modern approach to data management?
              How can agencies make the move toward this approach?
              Roadmap to Managing Data from Emerging Technologies for Transportation
              Supporting tools
   Guidebook for Managing Data from Emerging Technologies for Transportation              2
         Project Background and 
         Objectives
          Background:                                         Research Objectives:
          New, big, and varied datasets are available to     Develop a framework for managing data from 
            transportation agencies at an increasing pace.       emerging technologies, including data from 
          These data have tremendous potential to               connected and automated vehicles and data 
            offer new insights to transportation agencies.       linked to new mobility initiatives. 
                                                             Outline a process for applying this framework 
            The volume, speed, and granularity of these 
            data are unprecedented and will                      to help agencies incorporate these data into 
            fundamentally alter the transportation sector.       the decision-making process.
   Guidebook for Managing Data from Emerging Technologies for Transportation                                  3
          Transportation Agency Challenges to 
          Managing      Data from Emerging Technologies
            •  Reliance on traditional database management systems.                   • Experience a loss of control to vendors over 
               Data from emerging technologies are too large, too                        data, technology, and service agreements.
               varied in nature, and will change too quickly to be                    • Are unequipped to handle this level of big 
               handled by these traditional data systems.                                data at an organizational level.
            •  Struggle to break down business unit and data silos.
            •  Do not fully recognize the value of big data or the 
               eminent need to ready for it.                                                      “Our big data issues are 
            •  Do not fully understand the uses and benefits of cloud-                    straightforward, we don’t have the 
               based architecture conducive to handling data from                             technology, money, or skills.”
               emerging technologies.                                                                     – CITY DOT
            •  Have difficulty hiring and retaining modern data 
               management professionals.
    Guidebook for Managing Data from Emerging Technologies for Transportation                                                                    4
                                                          Big Data is More Than a Buzzword or Simply “Lots of Data” 
                                                          •  Big data may refer to data sets that are so vast and complex that they require new 
                                                             and powerful computational resources to process.1
                                                          •  Big data may encompass all the non-traditional strategies and technologies needed 
                                                             to gather, organize, process, and generate insights from large datasets.2
                                                          •  Big data is an approach to generating knowledge in which advanced techniques are 
                                                             applied to the capture, management, and analysis of very large and diverse volumes of 
                                                             data – data so large, so varied, and analyzed at such speed that it exceeds the 
       What is Big                                           capabilities of traditional data management and analysis tools.3
                                                          •  Big data is a term that describes the large volume of structured and unstructured data 
                 Data?                                       that inundates a business on a day-to-day basis. But it’s not the amount of data that’s 
                                                             important. It is what organizations do with the data that matters. Big data can be 
                                                             analyzed for insights that lead to better decisions and strategic business moves.4
                                                          •  Big data is a new attitude by businesses, non-profits, government agencies, and 
                                                             individuals that combining data from multiple sources could lead to better decisions.5
                                                             1 Big Data. Dictionary.com, 2019.
                                                             2 Ellingwood, J. An Introduction to Big Data Concepts and Terminology, Digital Ocean, 2016.
                                                             3 Burt, Cuddy, Razo. Big Data’s Implications for Transportation Operations: An Exploration, USDOT, 2014. 
                                                             4 What is Big Data? What is Big Data, SAS, 2019.
                                                             5 Press, G. 10 Big Data Definitions - What's Yours? Forbes, 2014.
                                                                                                                                                  5
               Traditional vs Modern Big Data 
               Management
       Managing data from emerging 
       technologies requires a complete 
       paradigm shift. These data cannot 
       be handled simply by adding more 
       hardware or processing power. 
       The nature of the data demands 
       an updated approach.
     Guidebook for Managing Data from Emerging Technologies for Transportation                                                                                             6