The #BigData Evolution and Revolution in 2017

biodata revolution evolution

Big data, a buzz word of overloaded information, gathers a set of technologies and practices for storing large amount of data and analyse in a blink of an eye. Nowadays, Big data is shaking our ways of doing business and the ability to manage and analyse this data depends on the competitiveness of companies and organisations. The phenomenon of Big Data is therefore considered one of the great IT challenges of the next decade.

 

4 major technological axes are at the heart of the digital transformation:

 

  • Mobile and Web: The fusion of the real and virtual worlds,
  • Cloud computing: The Web as a ubiquitous platform for services,
  • Big Data: The data revolution,
  • Social empowerment: The redistribution of roles.

Interconnected and feeding each other, these 4 axes are the foundations of digital transformation. Data, global or hyperlocal, enables the development of innovative products and services, especially through highly personalised social and mobile experiences. As such, data is the fuel of digital transformation.

The intelligent mobile terminals and the permanent connectivity form a platform for social exchanges emergence new methods of work and organisation. Social technologies connect people to each other, to their businesses and to the world, based on new relational models where power relations are profoundly altered. Finally, cloud computing makes it possible to develop and provide, in a transparent way, the information and services needed by users and companies.

According to Eric Schmidt, Chairman of Google, we are currently creating in two days as much information as we had created since the birth of civilisation until 2003. For companies, the challenge is to process and activate the available data in order to improve their competitiveness. In addition to the “classical” data already manipulated by companies and exploited by Business Intelligence techniques, there is now added informal data, essentially stemming from crowdsourcing, via social media, mobile terminals and, increasingly via the sensors integrated in the objects.

 

Why Big and why now?

 

3 factors explain the development of Big Data:

    • The cost of storage: the latter is constantly decreasing and is becoming less and less a relevant criterion for companies. Cloud computing solutions also allow for elastic data management and the actual needs of enterprises.
    • Distributed storage platforms and very high-speed networks: with the development of high speed network and cloud computing, the place of data storage is no longer really important. They are now stored in distinct, and sometimes unidentified, physical locations.
    • New technologies for data management and analysis: among these Big Data-related technological solutions, one of the references is the Hadoop platform (Apache Foundation) allowing the development and management of distributed applications addressing huge and scalable amounts of data.

 

These 3 factors combined tend to transform the management and storage of data into a “simple” service.

 

Sources of Data: 

 

To understand the phenomenon of Big Data, it is interesting to identify the sources of data production.

 

    • Professional applications and services: these are management software such as ERP, CRM, SCM, content and office automation tools or intranets, and so on. Even if these tools are known and widely mastered by companies, Microsoft has acknowledged that half of the content produced via the Office suite is out of control and is therefore not valued. This phenomenon has experienced a new rebound with the eruption of e-mail. 200 million e-mails are sent every minute.
    • The Web: news, e-commerce, governmental or community-based websites, by investing the Web, companies and organizations have created a considerable amount of data and generated ever more interactions, making it necessary to develop directories and search engines, the latter generating countless data from users’ queries.
    • Social media: by providing crowdsourcing, Web 2.0 is at the root of the phenomenal growth in the amount of data produced over the past ten years: Facebook, YouTube and Twitter, of course, but also blogs, sharing platforms like Slideshare, Flickr, Pinterest or Instagram, RSS feeds, corporate social networks like Yammer or BlueKiwi, etc. Every minute, more than 30 hours of video are uploaded to YouTube, 2 million posts are posted on Facebook and 100,000 Twitter tweets.
    • Smartphones: as the IBM specifies, the mobile is not a terminal. The mobile is the data. There are now 4 times more mobile phones in use than PCs and tablets. A “standard” mobile user has 150 daily interactions with his smartphone, including messages and social interactions. Combined with social media and Cloud Computing services, mobile has become the first mass media outlet. By the end of 2016, Apple’s App Store and Google Play had over 95 billion downloaded apps.
    • IOT: mobile has opened the way to the Internet of Things. Everyday objects, equipped with sensors, in our homes or in industry, are now a potential digital terminal, capturing and transmitting data permanently. The industrial giant General Electric is installing intelligent sensors on most of its products, from basic electrical equipment to turbines and medical scanners. The collected data is analysed in order to improve services, develop new ones or minimise downtime.

 

Data visualization:

 

An image is better than a big discourse … Intelligent and usable visualization of analytics is a key factor in the deployment of Big Data in companies. The development of infographics goes hand in hand with the development of data-processing techniques.

 

The data visualization allows to:

 

    • show “really” the data: where data tables are rapidly unmanageable, diagrams, charts or maps provide a quick and easy understanding of the data;
    • reveal details: data visualization exploits the ability of human view to consider a picture as a whole, while capturing various details that would have gone unnoticed in a textual format or in a spreadsheet;
    • provide quick answers: by eliminating the query process, data visualization reduces the time it takes to generate business-relevant information, for example, about the use of a website;
    • make better decisions: by enabling the visualization of models, trends and relationships resulting from data analysis, the company can improve the quality of its decisions;
    • simplify the analyzes: datavisualizations must be interactive. Google’s Webmaster tools are an example. By offering simple and instinctive functionality to modify data sets and analysis criteria, these tools unleash the creativity of users.

 

Big Data Uses: 

 

The uses of Big Data are endless, but some major areas emerge.

 

Understand customer and customize services

This is one of the obvious applications of Big Data. By capturing and analyzing a maximum of data flows on its customers, the company can not only generate generic profiles and design specific services, but also customize these services and the marketing actions that will be associated with them. These flows integrate “conventional” data already organized via CRM systems, as well as unstructured data from social media or intelligent sensors that can analyze customer behavior at the point of purchase. Therefore, the objective is to identify models that can predict the needs of clients in order to provide them with personalized services in real time.

 

Optimize business processes

Big Data have a strong impact on business processes. Complex processes such as Supply Chain Management (SCM) will be optimized in real time based on forecasts from social media data analysis, shopping trends, traffic patterns or weather stations. Another example is the management of human resources, from recruitment to evaluating the corporate culture or measuring the commitment and needs of staff.

 

Improve health and optimize performance

Big Data will greatly affect individuals. This is first of all due to the phenomenon of “Quantified Self”, that is to say, the capture and analysis of data relating to our body, our health or our activities, via mobiles, wearables ( watches, bracelet, clothing, glasses, …) and more generally the Internet of the Objects. Big Data also allow considerable advances in fields such as DNA decoding or the prediction of epidemics or the fight against incurable diseases such as AIDS. With modeling based on infinite quantities of data, clinical trials are no longer limited by sample size.

 

Making intelligent machines

The Big Data is making most diverse machines and terminals more intelligent and more autonomous. They are essential to the development of the industry. With the multiplication of sensors on domestic, professional and industrial equipment, the Big Data applied to the MTM (MachineTo Machine) offers multiple opportunities for companies that will invest in this market. Intelligent cars illustrate this phenomenon. They already generate huge amounts of data that can be harnessed to optimize the driving experience or tax models. Intelligent cars are exchanging real-time information between them and are able to optimize their use according to specific algorithms.

Similarly, smart homes are major contributors to the growth of M2M data. Smart meters monitor energy consumption and are able to propose optimized behaviors based on models derived from analytics.

Big Data is also essential to the development of robotics. Robots are generating and using large volumes of data to understand their environment and integrate intelligently. Using self-learning algorithms based on the analysis of these data, robots are able to improve their behavior and carry out ever more complex tasks, such as piloting an aircraft, for example. In the US, robots are now able to perceive ethnic similarities with data from crowdsourcing.

 

Develop smartcities

The Big (Open) Data is inseparable from the development of intelligent cities and territories. A typical example is the optimization of traffic flows based on real-time “crowdsourced” information from GPS, sensors, mobiles or meteorological stations.

The Big Data enable cities, and especially megacities, to connect and interact sectors previously operating in silos: private and professional buildings, infrastructure and transport systems, energy production and consumption of resources, and so on. Only the Big Data modeling makes it possible to integrate and analyze the innumerable parameters resulting from these different sectors of activity. This is also the goal of IBM’s Smarter Cities initiative.

 

In the area of ​​security, authorities will be able to use the power of Big Data to improve the surveillance and management of events that threaten our security or predict possible criminal activities in the physical world (theft, road accidents , disaster management, …) or virtual (fraudulent financial transactions, electronic espionage, …).

#CloudComputing: Fix The Present Before You Plan The Future

Cloudcomputing

Cloud computing is leading to a major transformation in the terms of digital technology by companies in all economic sectors. The associated challenges relate not only to activity and job creation among digital players, but also to a competitive gain that can be realized by all user companies.

 

The cloud computing model consists of providing remote and on-demand computing resources, infrastructure, platforms or application software. The advantages in terms of cost reduction and ease of access lead to this rapid adoption, which results in a gradual but decisive change in the information systems, activities and related markets.

 

However, complexity and lack of integration is slowing down companies’ adoption of the cloud, according to a study conducted by Oracle on the EMEA area. The wide gap between central IT and the rest of the organization is directing many companies towards a bad approach of the cloud.

 

While many European companies are adopting the Cloud Computing, nearly half of them are facing difficulties due to increased integration costs and data storage. One of the main reasons for this situation is that more than 60% of a company’s total IT spending is now directly managed by the different business units, instead of IT department, which prevents companies from benefiting from cloud services to which they subscribe to. To avoid these problems, IT department must be the one responsible for providing the funds to keep other departments running. Because the budget is an important tool for identifying and executing the IT initiatives that are crucial to each department, therefore it should be well discussed between IT department together with CIOs.

 

Study also revealed that organizations continue to finance their IT investments without taking into account potential revenue and innovative projects: 2/3 decision-makers claim that funding their IT is too traditional and penalizes innovation, and 1/3 decision-maker admit that the IT funding models of their organization are hindering IT innovation. As IT budget can be divided across various categories, depending on the complexity and sophistication of your company/department and its structure, it must reflect benefits of IT strategy. For example, if you’ve been communicating a strategy of migrating to the cloud and highlighting the operational savings, you should reflect those advantages and use them as justification for budget allotment.

 

Companies need to rethink their IT financing models and undertake a profound cultural transformation in order to fully exploit all the benefits of the cloud. 33% of respondents say that an inadequate model of IT funding is currently penalizing their business. 33% are also convinced that their company’s IT culture is insufficient for the cloud age. As a result, 72% of respondents say that a new cloud financing model will allow IT to offer more cloud services to the company, and 70% say it will allow the company to reduce its costs.

 

Problems that companies are facing in cloud computing adoption are less about technology but it’s about the difficulties of synchronization between the different business units. Managers of each department are increasingly making cloud purchasing decisions without involving the CIO or the IT department advice, especially because these purchases are very easy to make. So to be successful with digital business transformation and optimization, CIOs and leaders must brainstorm and communicate the strategy to allow IT spending and functional resource costs to be connected to business processes, outcomes and goals. By developing multiple views of the IT budget and resource allocations per department they can provide a better IT service supply on demand.

Managing Data Traceability: Impact and Benefits

Data is at the heart of digital transformation. A company can’t support data lacking integrity if they aim to advance in their digital transformations initiatives.  The integrity of the data lies primarily in the confidence that users can have in the latter. Most of this trust rests on the traceability of data. In the absence of traceability, it is not possible to know if these data are trustworthy.

 

Data traceability is a concept that companies have been trying to understand for some time.  You might be asking for which reasons do today’s companies need more traceability? Well with a large amount of data from unmanaged external sources (sensors, data streams, Internet of objects), it is essential, for companies, to monitor these data when they are collected, processed and moved to be able to use them effectively. Digital transformation requires higher levels of data integrity. Indeed, companies need to have better data, which can be a basis they can trust.

Data traceability

Previously, data traceability was based on two dimensions: “where” and “how”. The need for better analysis and exploitation of the data leads to new demands and extends the definition of data, adding the following dimensions: “what”, “when”, “why” and “who”. Faced with these new requirements, it is necessary to master the bases of the primary components of the type: “where” and “how, especially as regards the impact and the value to be realized.

 

The “where” component of traceability focus on the origin of the data. The “how” component focus on how the data source was manipulated to produce its result. It is also possible to refine these two types of dimension via their level of granularity: “low” granularity and “high” granularity. The “where” component at the “low” granularity level focus on defining an upstream output dataset at the point of consumption to understand which dataset have been used to produce a result. The “how” component of the “low” granularity level focus on the transformations applied to the set of data source to produce the output dataset. On the other hand, “high” granularity level of traceability is concerned with the values ​​of data in the “low” level granularity: instead of where they were created and how they were modified to produce the result.

 

An example will better illustrate the types and granularities of traceability. Let’s take an accounting report, showing the total amount paid to suppliers over a given period. The “where” component of the “low” granularity level would trace all output data from the source invoice to the supplier tables from the accounting application. The traceability component “how” of “low” granularity level would look at how the supplier and billing tables were linked together with the calculation functions that were performed on the billing table to produce the total amount paid to each supplier. Traceability “where” at the “high” granularity level could (to search for the amount paid to a vendor) trace back to the invoices provided by the supplier. In order to take interest in the entire process, traceability at the “high” granularity level could also link to the original request: the purchase order, the receipt operations, in addition to the payment approvals.

 

Benefits of using data traceability

 

Declined in many forms, traceability can provide many benefits in terms of impact and added value to the companies that implement it. Such as:

  • Governance: Ensure the traceability of upstream data to provide owners and data sources with quality and access control results. This will allow data owners to manage their procedure in addition to downstream traceability (integrated with a corporate glossary, data traceability can allow data managers to control current definitions and understanding of terms and fields of data).
  • Conformity: Provide regulatory authorities with information to govern data sources, users and their behavior.
  • Change Management: Enabling users and developers to understand the impact of modifying certain data on downstream systems and reports.
  • Development of Solutions: Improve design, testing and deliverables of better quality. This is achieved through the sharing of traceability metadata, glossaries, and relationship among distributed development teams.
  • Storage Optimization: Provide as an input to archive decisions and provisions, an overview of the data being accessed and indicate where, how often and by whom access is permitted.
  • Data Quality: Improvement of the quality scores defined by the application of business rules and standardization on data, added to the metadata population as input of algorithms and decision making.
  • Problems Resolution: Helps in the analysis of root causes in repair-type processes.

 

Traceability also brings a deeper advantage such as focus on the changed values ​​of the basic data entities that are shared between processes, services, and applications. For example, the impact that a change in, position, service, address or even the employer of a contact might have on the marketing, business or maintenance service of a business. According to the “U.S. Bureau of Labor Statistics,” an employee has on average 11 different employers throughout his career. Taking into account the speed at which US residents move and change their professions each year, the potential change in baseline data may be important in light of the adequacy of these statistics to the population of basic data within a company. The ability to collect, validate, distribute and track these changes in a timely manner could lead to better protection of existing revenue streams and the ability to capitalize on new revenue in B2B or B2C business relationships.

So, the companies which take advantage of traceability, are able to find data faster and are better able to support security and privacy requirements.

IT Challenge n°2: Rise of new Partnership Models

 

IT Partnership Models

2020 companies will be totally interlinked organizations within an ecosystem in which new strategic partnerships and associations will be formed, as well with customers, suppliers and competitors!

 

A profound transformation of the ecosystem

The growth of value creation is a major trend in digital era. We witness more and more companies opening up, thanks to the multiplication of the interactions allowed by cloud, data repositories, connected objects … This condition requires companies to rethink their business partnership strategies within their ecosystem in order to succeed in the age of digitization.

This ecosystem is very extensive, with an interesting diversity of actors, such as, GAFA (Google, Apple, Facebook, Amazon), start-ups, innovative SMEs, communities, customers, employees, self-entrepreneurs, suppliers, public and local authorities and political institutions… In the era of “co-something”, a company can no longer succeed alone in its market, particularly because of the rapid emergence of new business models, competitors “out of nowhere” and an accelerated renewal technology.

 

The challenges: anticipate and ally

Controlling the ecosystem depends on anticipating the evolution of it’s different actors, to be noted that actors in the traditional IT are not necessarily those of the current ecosystem, nor, of tomorrow. Some will disappear or merge, others will emerge, many will become partners.

Establishing a good relationship with the right partner, which can be a supplier, requires joint sharing of opportunities and risks, commitment to common goals, and shearing value. And this sharing of value aims to bring something new and positive to the partners, and ultimately to help them grow. Strategic partnerships can be established when there are common objectives for value creation. With this perspective, the partnership is strategic and is quite different from the traditional customer-supplier relationship (even major), in which the parties are bound by a contract for the providing services.

The objectives of each party must be the same and the balance of the relationship arises precisely because of that different but valuable opinions and ideas.

 

Challenges: Negotiation and Confidence

  • Collaborate: one of the typical challenges of partnerships will be to manage the paradox between internal resources (including CIOs) that are experiencing difficulties and struggles, collaborating and, on the other hand, the market, which requires close collaboration to better innovate.
  • Dialogue: companies are confronted with a cultural interoperability challenge in order to engage all the actors involved, even if they do not share a common language.
  • Establishing trust: a partnership relation is always based on trust. Thus, it is not a question of “collaborating to collaborate”, but of collaborating to win together, in order to create communities that engage clients and collaborators.

R-Link is the result of a long-term partnership between Renault and Tom-Tom. R-Link is an integrated multimedia tablet, driven by a tactile control or an advanced voice command. It combines, the various functions related to multimedia in the car such as, navigation, radio, telephony, messaging, well-being, eco-driving. Renault’s interest in combining with Tom-Tom was to increase the value for its customers, to know them better and to improve the level of service.
This example illustrates the idea of a service platform: Renault added services to its products by developing the customer experience.

 

To conclude, I’ll say that the success of these new partnership model depends only on the business taking much greater role in designing, building, and exploiting the technology, platforms, and data it needs to succeed. Overcoming challenges of traditional IT management is a step forward of bringing IT closer to its true mission and succeeding in all IT collaborations.

#MachineLearning: How #PredictiveAnalytics reinvents Customer’s Satisfaction

Billions and trillions of data is collected on customer behavior from the huge platform called internet. To these are added valuable information gathered by every organization from different sectors. In this mine of information, Machine learning, pursues an ultimate goal: to better understand customers in order to offer them the best experience possible by offering them the product or service most likely to their need. It’s analytical power and the advance in artificial intelligence allows companies to take advantage of the wealth of data they collect.

At this point we all know that #bigdata is worth nothing, nada, without proper decryption. This is where machine learning or “automatic learning” comes into action. With its power of analysis, this field of artificial intelligence extracts the valuable information from the mass data. In other words: it enables to turn the lead into gold by simplifying the life of the customer and improving its satisfaction thanks to the precise analysis of its way of purchase.

 

Artificial Intelligence: algorithms and insights

Since its first use in the general public in the late 1990s, the machine learning have never stopped to make talk about it. Its recent victory was in March 2016 via AlphaGo, the software of Google, against the legendary Lee Sedol. We’ve witnessed AlphaGo’s most notable examples of deep learning, which was, the ability of a machine to independently analyze sums of data with an extremely high level of performance.

If such technological power remains exceptional, all of us daily experience the small machine learning without knowing it. How? Well, just surf on Amazon, LinkedIn, Spotify or even Netflix to see these platforms automatically offer suggestions according to their precise taste. These associations of ideas remain pertinent on subjects as fine as the interest for a film, a song, a service or a cross purchase. It is a much less superficial intelligence than it seems but with concrete results.

 

From big data to automatic learning

Well-resourced with quality data, the algorithm analyze deeply in the vast meadows of digital world. They cross distant data from each other to reveal information never brought to light. These algorithms bring us the astonishing results which a human mind would have swept away. For example, in a customer journey, deep learning allows to discover that the intention of purchase can be correlated with an action at precise moment of purchasing action. With automatic learning, one can therefore target with precision every important thing that human understanding can escape.

 

Machine learning: better tracking of customer routes

According to Salesforce’s state-of-the-art survey published in 2016, customer engagement is a top priority for organizations. Customer satisfaction is the main reason for success, even surpassing revenue growth and the acquisition of new customers. In this context, Machine learning is thus a major ally.

From an operational point of view, most of the machine learning applications used today are subject to a pre-learning phase. A large amount of data is thus processed, during algorithm design, to better guide the search and automate more easily the answers that will be offered to online surfers. It comes to deal with a combination between human intelligence and artificial intelligence. The goal still to be reached, for each organization, is a user experience that is as simple and fluid as possible. The machine learning has already made it possible to take a major step forward thanks to the ultra-segmentation of the profiles for a refined follow-up of the customer routes.

 

Sharing Data: the essence of war

In order to function at full capacity, machine learning must benefit from first-class information. How it’s possible? By adapting an omnivorous diet. Depending on the project, companies use the information they collect through cookies, geolocation, social networks, loyalty programs (which typically collect data on age, location, purchase history …).

Contrary to popular belief, consumers are rather inclined to share their data, but not at any price. This is evidenced by the Columbia Business School’s “What is the future of data sharing” study conducted by the Columbia Business School Center for Global Brand Leadership in 2015 with 8,000 Internet users in the United Kingdom, the United States, Canada, France and India. “Consumers are much more knowledgeable about the issue of data sharing than we originally suspected. According to our study, one of the determining factors in the decision to share them is trust in the brand, “says Matthew Quint, director of the Center for Global Brand Leadership. Researchers at Columbia Business School have come to the conclusion that more than 75% of Internet users more readily share their data with a brand they trust.

 

Customer data: Give and Take

Beyond trust, the sharing of information is based on a give-and-take approach. According to the same Columbia Business School study, 80% of consumers agree to share confidential information in exchange for a reward. It must be a “valuable offer, but this value must be clearly defined and easy to understand to hope for the best possible return on investment,” says Matthew Quint. Young consumers would be more favorable than their elders to concede their personal information. What promises beautiful days to machine learning.

 

All the above points ends on the same conclusion that organizations can get a better understanding and add a new layer of intelligence on their customers behavior by using predictive analysis.

#InternetOfObjects and the Emerging Era of #CloudComputing

IOT CLOUD

Big data and connected objects represent an important source of economic growth according to numerous studies. They open the possibility to connect people or objects in a more relevant way, to provide the right information to the right person at the right time, or to highlight information that is useful for decision-making. Allied to Big Data, connected objects give professionals new opportunities to better understand customer needs and better satisfy them.

 

According to McKinsey, the overall economic potential of the IoT universe could be between $ 3,900 billion (US ‘trillion’) and 11,100 billion per year by 2025! So with 30 billion connected objects by 2020 it’s now necessary, more than ever, to rethink the use of Cloud.

 

The explanation of this boom?
Connected objects are already very widespread and are gradually taking over all sectors. The general public sees it as a way to improve everyday life, while companies are already using it to control and improve industrial processes and propose new services. Cities and vehicles are becoming smart by using different types of sensors.

 

Nearly all manufactured goods entering the market – vehicles, equipment for energy or water supply, health sector equipment, scientific and technical research facilities, machine tools and robots, etc. – all are bound to be connected and, for a good part, to be interconnected.

 

We are only on the premises but very well equipped with advanced technologies, the only thing to do is to imagine their great usage that will respond to every real expectations and will bring real added value. This ability to make our environment much smarter is linked to sensors, to the data collected by these sensors and to the speed of processing of this data. The triangle of Connected Objects, Big Data and Cloud will become essential to transform this universe of connected objects into intelligent systems.

 

Future of IOT Data 
The continuous flow of data generated by IOT is challenging the networks. All of these billions of objects that can be interconnected via the Internet are accelerating the real tsunami of announced data. The cloud is a simple and flexible way to deal economically with this mass of data that will continue to grow with time and new uses. And to cope with this huge data, the computing power will have to be adjusted. With the successful adoption of IoT, manufacturers will work on new systems architectures, especially those that are “hyper-integrated”, “hyper-convergent”, and can bring very high performances.

 

Cloud, indispensable for the development of the internet of objects
Connected objects are synonymous with capturing very large masses of valuable data. The gathered information via IoT will have to be stored, transmitted and analyzed for which the choice of Cloud infrastructure is the most appropriate method. Firstly because of the flexibility afforded by this type of offer only a Cloud solution allows real-time adaptation of the infrastructure capacity according to the level of demand. A flexibility for the management of all the connected objects devoted to knowing peaks of load and allows connected devices to interact with powerful back-end analytic and control capabilities. 

Furthermore, this flexibility can play more decisive role for commercial success, a situation in which it is essential to adapt its infrastructure quickly to meet demand. A necessity that affects the companies of moderate sizes seeking to contain their investments in technical infrastructures.

A flexible cloud service for connected devices can facilitate critical business decisions and strategies process by allowing you to connect your devices to the cloud, analyze data from those devices in real time, and integrate your data with enterprise applications, web services etc.

 

New skills and infrastructure needed
Applications linked to IOT are limited only by the human imagination. From automotive to home automation, to medical and healthcare industry, entertainment and education, IOT is pervasive and growing rapidly and transforming all economic sectors. To operate these innovative devices, it will be necessary to develop applications capable of collecting and processing the data that they will generate. The manufacturers of connected objects and the service providers responsible for the management of these applications must therefore provide themselves with appropriate skills and infrastructures.

Ten tips to avoid your #BigData project failure

If you’ve shown some interest to this week blog post, we assume that it’s because you want to avoid making mistakes that could cause your Big Data project failure.

 

Here are some points you must consider:

 

 Begin any Big Data project with a data review and classification process. In order to design a powerful storage architecture you must determine whether data is structured, unstructured, qualitative or quantitative. It is also a good idea to estimate the growth of data based on past trends and future strategies.

 

Create a simple overview of how data flows within your organization. Having a simple diagram showing where data is created, stored, and circulated is useful when it comes to work within a group. Putting everyone on the same wavelength can help you avoid misunderstandings that are expensive.

 

Consider future data storage requirements based on the success of the Big Data project. Big Data projects may reveal new information or require you to change business processes. Information from the project may in other hand require additional storage capacity, resulting in exponential growth in capacity requirements. Always think in long term.

 

Be flexible. Many projects are based on both scale-up and scale-out storage technologies. Each organization and project is unique. The choice of a storage technology must be based on the objective to be achieved and not on a particular technical architecture. Many vendors offer scale-up and scale-out products that can work together.

 

 Data storage requirements may increase, but consider to move automatically low-visited data to a cheaper, slower storage device. Removal is also a considerable option in the long term. Regardless of where the data comes from, where it is processed and where it is stored, it always has a useful life. Deciding to delete data is a complex task, but this can enable you to realize huge savings in long term. Automatically moving data to a slower storage device is an easier option that always involves great benefits.

 

Ask providers what will happen when you reach capacity limits or theoretical performance limits. Even if you start with a small Big Data project, it will surely get bigger with time. Understanding how the chosen technology can evolve will help you avoid unpleasant surprises in the years to come.

 

Be prepared for the worst. Even the simplest machines can eventually break down or black out. Ask your vendor what would happen if various elements of the storage platform were to fall. A well-designed system should never have a point of failure.

 

Create a quota system early in the project to prevent future management problems. IT projects tend to occupy all available space if they are not controlled. Quotas allow you to define how much space each user or project can access. Assign the responsibility for managing this capacity to an entity responsible for this data or define a policy.

 

Always involve IT security experts in your Big Data projects. Digital data has value. Even though the Big Data project involves only one research group, the IT security team must be involved from the start, so that security remains at the heart of the project.

 

Don’t forget to take management time into account when it comes to calculate storage costs. Total storage costs must include the time required to supply and manage the platform. A resilient, highly automated system that does not require a full-time administrator saves far more money in the long run than a less expensive hardware that requires a lot of manual labor.

 

I really hope these tips will help you in your project planning. In case you’ve any question or need an advice please feel free to write us and our #DataHeros will contact you asap!

 

#BigData: Jobs and Key Skills Businesses Need

Data is an organization’s most valuable asset, and the best way to nurture and protect it is through a governing body that is responsible for setting consistent data standards for the organization. The production of data is expanding at an astonishing pace. Experts at EMC point to a 4300% increase in annual data generation by 2020. Due to this digital Data increase, it is, now more than ever, essential for companies who intend to take full advantage of the real value of big data, to recruit new talent to improve their productivity.

big data jobs and skills

Rare profiles, with degrees from different backgrounds have the task of extracting this unstructured data, to transform them into beneficial actions and operations for the company. Companies working with big data are already facing challenges when it comes to finding and hiring the best possible talent. Recognizing this growing need, these companies will have to recruit distinct profiles with knowledge of training and data-oriented diplomas.

 

But what are these jobs?

 

Chief Data Officer (CDO): 

He is the Director of the data, the #DataGuard. He leads a team that specializes in the acquisition, analysis and data mining. Its main function is governance of his team for the supply of the most interesting and valuable data for the interest of the company. Based on statistics, computing and digital knowledge, he gives insights to each department such as, marketing, human resources, engineering, quality department, accounting and management. Graduation from engineering school is required, as well as skills and experience in the fields of management, IT and marketing are needed.

 

Business Intelligence Manager: 

His job is to facilitate the decisions of the CDO. He use new technologies to develop dashboards, reporting tools, in order to integrate the computer system and make them available to company users. This profession requires a solid knowledge of English, computer and data management. Just like CDO, graduation from engineering school is required.

 

Data Scientist: 

He is responsible for the collection, processing, evaluation and analysis of big data to optimize the company’s strategy. His role is to create for the Company, algorithms that produce useful information, particularly in order to offer customers the products they want. These profiles combines management, IT and statistics skills. They master the techniques of data mining, as well as technologies and IT tools databases such as Hadoop, Java, MapReduce, Bigtable, NoSQL … A degree from engineering schools is essential.

 

Data Analyst: 

He works with statistical tools and specialized computer technology to organize, synthesize and translate the information companies need to make better decisions. Data analysts guard and protect the organization’s data, making sure that the data repositories produce consistent, reusable data.Graduated from engineering school is required.

 

Le Data Miner: 

He is the “data excavator,” the Sherlock Holmes of the company’s data. His role is to find the information from multiple data sources to make them usable and useful for the company. He must have excellent computer, business and statistical skills. It is possible to become Data Miner from a computer or marketing Degree. He can potentially evolve into a Data Analyst and Data Scientist.

 

Master Data Manager: 

Data Manager acquires and organizes information from the company for their optimal use. He is an expert in database including, the reference data (related to supplier catalogs, customers, products, etc …) and structural metadata (related to regulatory standards and methods). He must ensure that these data are consistent and well organized according to defined business rules and properly integrated into the information system operated by the business teams.

 

Data Protection Officer: 

This person is the guarantor of keeping a record of all the processing operations on personal data carried out by the company where he works. This position could become mandatory in all companies with more than 250 employees. His challenge is to be informed of all data processing projects within company so he can input his upstream recommendations. It must not only bring together computer and law skills, but also strong communication skills. This is a new job of digital business that appears in a highly competitive environment where data protection issues is the heart of business and represents a major challenge for the economy.

 

The big data experts are both very rare and in high demand. They are found mainly in large groups such as banking, insurance and finance, or in the operators that store and process data such as data centers, internet service providers and web hosts. But the regulation of rapidly changing data, business data and opportunities are multiplying, so all companies will be surrounded, near and far, with similar profiles to those presented above.

So if you are looking for a job, recruit talent, or simply want to learn more about these new jobs, feel free to contact us on LinkedIn: https://www.linkedin.com/company/xorlogics