Incorta Fundamentals: Building Business-Ready Analytics in the Cloud

Transcript:

Ahmed Tarek: hey everyone just wait for one or four minutes.

Ahmed Tarek: To enable more people to join them really stop the lap.

Ahmed Tarek: Yes, so I think we can stop right now so good afternoon everyone Thank you Thank you so much for your time today and for joining us in that's all Apps for in court.

Ahmed Tarek: Today we'll talk about the put the fundamentals and how we can build a business spreading the analytics and the cloud over in code itself.

Ahmed Tarek: Let me first introduce myself and one other stuff inside the webinar, so this is me as I thought i'm leaving the presets for the Middle East, I have Mike lanes David and Martha.

Ahmed Tarek: Actually, David a martyr can support you, through the Q amp a chat section, if you have any questions any issues or something like that you can handle you.

Ahmed Tarek: Besides that will share with you, through the chat as well at the lab guide that has the full detailed, step by step of what we will do in this lab as well.

Ahmed Tarek: And it can be you reference if you missed something would miss anything during the lab I will do everything step.

Ahmed Tarek: with you, so you can replicate that with me, along with me while i'm doing the practice, I will be in a slow mode to make sure that you are on track with me alone, the lab as well, you have any questions or any song sing need more clarification you can bring us on the q&a section.

Ahmed Tarek: yeah so before going to the agenda, you can set to encode that cloud cluster for you by actually going to that the go for the website of the quarter.


Ahmed Tarek: and go to.com you can find here start for free.

Ahmed Tarek: So it will be giving you a link to go for the cloud in photo COM, you can register and the sign up using your business email, and once you put you immediately will receive.

Ahmed Tarek: A confirmation to activate your email and confirm your password and after that you will find your cluster over includes the cloud up and running and began opening the platforms, through your a web application.

Ahmed Tarek: Going back to the part of the presentation.

Ahmed Tarek: So we'll start with what is the quarter to give you a brief and the technical details about the product, the approach the architecture of.

Ahmed Tarek: Equal the platform and, in general, before going to the platform itself and navigating across the platform.

Ahmed Tarek: and showing you the full data pipeline, how we can build a full data pipeline through important platform from connecting to the data loading the data, creating a full dashboard is multiple insights and sharing that insights with external users or internal users across.

Ahmed Tarek: platform or across your organization.

Ahmed Tarek: As I mentioned, so every step from this agenda from starting from the lab itself, you can do the steps with me, along with me while i'm doing that, and you can they fear as well.

Ahmed Tarek: For the lab guide and if you have any issues for accessing the lab guide, so you can Ping us also through the chat and the team will handle that for you.

Ahmed Tarek: yeah so let's just talk about the modern data architecture that's existing right now in most of the organization.

Ahmed Tarek: So you can find the here from that architecture.

Ahmed Tarek: As either part you have multiple source systems, you can build your data lake or your data warehouse your data leak to define your data raw data zone to print or to extract data from these multiple systems.

Ahmed Tarek: And you need a data warehouse to be defined it as room to do that require modeling and putting the data reshaping the data into star schema snowflake schema based on the business requirements that you have so you start to build your modeling box.

Ahmed Tarek: And after that you need to build that aggregated level over that like a data marts to bring the data for the analysis and for the visualization tools for every department.

Ahmed Tarek: So actually this process is highly costly from a time perspective and from the resources perspective as well, and the from the money perspective.

Ahmed Tarek: Besides, that you are losing, I think, a very important part of your data, because you cannot keep your data from the source, where was a full visibility or full data accessibility.

Ahmed Tarek: You shouldn't mess, a lot of data, while you are moving from a hub to another hub across the data pipeline itself.

Ahmed Tarek: So as a time it's it's a key factor for delivering a critical business analytics the time as well, is very practical for the operational reporting itself.

Ahmed Tarek: So if you are talking about the finance team, the HR the supply chain team, they need to have a quick answers for the questions business questions or their ad hoc reports that they need to be generated in a quick way.

Ahmed Tarek: Not that only but actually they need to understand what's behind the numbers and what's behind these kpis or these insights, and this requires that you have a full data accessibility, that you mess here in the most the modern architecture world.

Ahmed Tarek: So what the approach that in code that is bringing right now.

Ahmed Tarek: So in kota is a unified data platform.

Ahmed Tarek: That giving you the ability to connect and in just directly to your the disruptions.

Ahmed Tarek: With our beacon technology, which is a virus data mapping, so you can in just in connect to the data sources directly.

Ahmed Tarek: And maps of them directly into in quarter engineer itself, so you don't need to reshape your data into another shape.

Ahmed Tarek: To build analytics or something like that, so you are now, you can deliver now 100% of the data to your business users, so in cold the scary thing are drying approach for the analytics.

Ahmed Tarek: So whatever or whenever any business question is come so your data, the data is available.

Ahmed Tarek: So you can build whatever that you need, whatever the business need from their perspective directly over in both because you have the full accessibility.

Ahmed Tarek: So in Code, the approach here is creating a joy a push for the analytics was having a direct connection and direct mapping, where the data sources itself.

Ahmed Tarek: And the ability of a quote as well, so you have you are ingesting 100% of your data so at the same time, because we are not missing any data, while moving the data, you can analyze 100% of your data as well.

Ahmed Tarek: And besides that include the has the box, that the bi capabilities that we have in the product from a visualization and good eating, vegetables and insights inside the product itself.

Ahmed Tarek: And you are giving the ability from the business users, that the like special platforms like tablo was the I excel or something like that to connect as well to encode so in court that's not a lot of technology it's an open technology using open standard.

Ahmed Tarek: lifers and technologies in the product and which will that inside Doc Dr itself.

Ahmed Tarek: So any third party or a visualization tool we can connect to include Messiah they quote the visualization part that the gold already has part of the product.

Ahmed Tarek: After that, what did a lot of data mapping that we have, or what are the benefits that we have across the line mapping, so people is going to model, the data and refine the data and do a transformations over the data.

Ahmed Tarek: To make sure that this will not affect the operational system, so the need to move the data in aggregate it we were songs like that need to make sure that the visualization tools are using.

Ahmed Tarek: can render the results of the output, so they cannot work what they're audited like it needed to be model, but the bar here of the direct mapping technology that.

Ahmed Tarek: Whenever we are talking about the number of things that you have in us so systems reading about hundreds and thousands of drawings whenever that's happening, the performance in in kota as it stands, the same while you have a one joy.

Ahmed Tarek: So it's a linear relation that I can say, between the time and the number of joining for the data data mapping that we have inside in quotes.

Ahmed Tarek: So, from a performance side from the complex doings and the complex relations that we have that you have your source systems important cancel that using the direct with the mapping.

Ahmed Tarek: The second advantage of the second feature that we have in the director mapping if the rich should meet the data map.

Ahmed Tarek: So, so why important that is connecting to the source system, it can inspect and detect the structure of the schema and distill that into in court, the Meta data.

Ahmed Tarek: And, as well as the twins and the religion itself, so we haven't made it should make a data map, so the system or the technology or the engine itself understand how each data point inside the system, a blank to get.

Ahmed Tarek: So, because of that, the engine is providing you with a smart rerouting feature.

Ahmed Tarek: So you don't need to think about how I should design the screen how it can optimize this query to generate this insights.

Ahmed Tarek: You don't need to think about that, because the engine, because the understand the bosses doing prophecy.

Ahmed Tarek: While you drag and drop the required, I mentioned that the measures to create a specific inside the query plan will be generated automatically.

Ahmed Tarek: The most effective Community that will be generated for marketing, so this is the main features, or the main bar feature that we have.

Ahmed Tarek: Beside or behind the device with a mapping technology that's giving you the ability that you can do directly with your source systems and map them directly into important platform.

Ahmed Tarek: So you don't need to build a lot of etfs cracks to bring your data as well, so just a one to one mapping and after that, if you would like to do.

Ahmed Tarek: A transformations you can do that inside the book itself so in kota is following the path of not the EPL approach the elt approach itself, as you would like to name it like that.

Ahmed Tarek: So inside the goal that you will see that flow, as I mentioned, this is a building a full data pipeline as encode that is a unified data platform.

Ahmed Tarek: So you can connecting to data sources, using the data with with data connections that we have inside the product, you can build or design your schema.

Ahmed Tarek: again after that build a business schema which is equally for the semantic layer and abstract layer for the business users and for the self service and export.

Ahmed Tarek: And at the end, build insights and vegetables over a quarter and sharing that with the stakeholders for web users within your organization.

Ahmed Tarek: So this slide is showing you the architecture and what's under the hope for encode the products and the different components that we have, and that will follow the same flow inside in code itself.

Ahmed Tarek: So let me year the pointer.

Ahmed Tarek: So the first component, that we have, as the data acquisition component which has the clothes, for having the connectors that enables you to connect to different school systems in court is providing more than 200 connectors that can enable you to connect to a different data sources types.

Ahmed Tarek: And after that the schema production will detect the schema the structure itself, and even the relationship that existing inside the data swoops if there are foreign keys are inside your school system itself.

Ahmed Tarek: And that doing a battle data load to load the data from these different data data sources, using the direct data mapping technology, which means that just a select statements to print or to extract data from your source systems into encode the storage.

Ahmed Tarek: In Incorta is a storing the data into a park a file format it's an open standard format it's the corner base and doing a good compression and excellent compression ratio as well, so.

Ahmed Tarek: Most of the cases, you will see the data size inside important storage will be less than that being in the school system itself and it's a calmer based and actually the calmer bates has a lot of adventures through the park people as well.

Ahmed Tarek: So we have the data now they're all data itself here in the parties to which physically stored on the desk and we have the Meta data that we talked about the individual Meta data that we have.

Ahmed Tarek: That includes all the metadata information needed for the engine itself to identify how each point in the system is linked together.

Ahmed Tarek: So, once the data is loaded right now so here again is a direct data mapping to bring you the ability for the data analytics component.

Ahmed Tarek: So the data analytics component, it includes multiple sub components here, the first one, our footprint in memory optimized analytics engine.

Ahmed Tarek: That can help you, you can build analytics when you can build insights and the doing that require aggregation.

Ahmed Tarek: Even was multiple joints across hundred million of records between different tables with a very powerful and very fast performance to be generated on the fly inside the memory of my century.

Ahmed Tarek: So you are just the drag and drop the word columns to generate a specific inside so then he did columns only.

Ahmed Tarek: for generating that inside will be moved to the memory, to build the required aggregation over inside the memory analytics engine itself.

Ahmed Tarek: So, because of that we are optimized, we are not moving all the columns to the memory just the needed for generating that really or for generating that insight that needed columns only will be moved to the memory to.

Ahmed Tarek: offer that, as I mentioned, with them in Code, the flow, the business views box or the business is schema bought.

Ahmed Tarek: This does semantically when opposite track the layer that you can bring different columns across different tables and even across different data sources in one view.

Ahmed Tarek: With a business labeling the business can understand, and you can give access for these business views to the business users that began to their self service analytics directly.

Ahmed Tarek: On top of that was applying that require security rooms over very low level data over the data itself or over the column part or even over the authorization from the access.

Ahmed Tarek: Port or disability parts for the different users, so, for example, I can have a same dashboard and different users can see different data based on the security rules that applied inside important south.

Ahmed Tarek: And we have our data visualization component that includes a lot of capabilities from the visualization part and we have our secret to face that enables any third party to read the data from encoder as an Boston glass.

Ahmed Tarek: database that is just an interface, we are providing for the external applications that can connect and read the data from importance of.

Ahmed Tarek: The last main component is the data enrichment part part of the product components as well.

Ahmed Tarek: Is this part, so as far as part of the product to cover the data enrichment box, so we have a zipline notebook is embedded inside the bathroom as well.

Ahmed Tarek: Is this part, so as far as part of the product to cover the data enrichment box, so we have a zipline notebook is embedded inside the bathroom as well.

Ahmed Tarek: So for the little interviews and for the data scientists, if you would like to build a custom transformation or a custom logic.

Ahmed Tarek: On top of the data that's loaded or that says stored inside in quarter storage, they can do that, using a concept of the notebook and the materialized views.

Ahmed Tarek: was different languages such as a spark Python spark our spark sequel scala scholar and autographs as well to build a custom logic to bell data science projects or machine learning projects and the good here that.

Ahmed Tarek: The output of this dragon tables or these materialized views are these data science algorithms the apples itself will be saved the physical again.

Ahmed Tarek: onto into import the storage itself as a new body fights so you can use the output for the analytics or for the visualization and you can use the apple to link it with with any other data source of any other people inside in court attempts.

Ahmed Tarek: and the last thing is the admin we have an admin ui it's all the cmc giving you the ability to configure your cluster because every component here can be horizontally scalable.

Ahmed Tarek: Why, you need to increase the required machine specs or increase the number of nodes needed to apply the high availability and everything related to the architecture itself of in court.

Ahmed Tarek: So that's a brief about have the data flow exactly happening and how the component, that we have inside the product is talking with each other.

Ahmed Tarek: So, again just a reminder, if you have any question, you can go directly the Q amp a section and Martin leave that can help you with that.

Ahmed Tarek: While and now, I will start and moving to the last part to start the lab exercise that we have.

Ahmed Tarek: And you can build actually every step, I will be in a more store mode, while i'm doing the steps to be with me, along with me with every step that i'm doing here, and you can refer to the guide the lab guide as well.

Ahmed Tarek: So let me go for.

Ahmed Tarek: Equal.

Ahmed Tarek: Well right, so this is a cloud cluster that I have over a quarter, so this is the homepage, this is the landing page while you are opening.

Ahmed Tarek: Your cluster, you will find this is more page the landing page that we have here.

Ahmed Tarek: it's a very simple page giving you the first step to connect to the data and start creating a data connection in giving you some of advanced materials like encode the Community it's better for you and I will.

Ahmed Tarek: Talk about it in the last slide after finishing the lab and give you also a link to the virtual hansel love coming sessions, that you can follow as well and that kind of tools or the knowledge base for encode the product itself.

Ahmed Tarek: So if we can go fast food start was a quick ui to that you can see, we can start from the right side, the first tab it's related to the security tab itself.

Ahmed Tarek: So here, you can find that different three taps the user step the group stop the whole stack so you can create multiple users, you can.

Ahmed Tarek: Actually, can create multiple groups, if you like, and you can assign those for each group in code that has a pre defined roles.

Ahmed Tarek: As you can see here that you can actually assigned to the creative groups and users would require security rules that you have a recommendation and import the has the ability as well to integrate with existing security systems to be things like active directory, and all that as well.

Ahmed Tarek: So the second tab is regarding to the data itself and start connecting to the data and it giving you varieties of the different data sources and the different contiguous that we have inside in quarter from a data science perspective from a database from application layer

Ahmed Tarek: For my data lake as well from web services from file systems and others like SAP year being Google sheets and customize things from connecting from school to beta or even building a new custom connector and the streaming data at the end by.

Ahmed Tarek: The second tab is the third tab is the schema part, while you are start to building or designing your schema after connecting to your source system and after that the businesses schema like what we mentioned.

Ahmed Tarek: In semantic layer or abstract layer for your complex systems.

Ahmed Tarek: And here is the scheduling part regarding how it can escape your my dashboard to be sent for a specific users on a daily basis or hourly basis was for a specific group.

Ahmed Tarek: And for the schema load to load to the schedule to log viewer schemas every five minutes to achieve the near real time analytics part.

Ahmed Tarek: And, as well as the little notification that, based on a specific kpis or a specific threshold, you need to.

Ahmed Tarek: be the state, the business stakeholders about the specific measure or a specific kpis that have a specific threshold or some very specific rules are defiled them, we are naming that as a little vacation or the alerts.

Ahmed Tarek: and the last thing for the schema notification So if you are doing a load for a specific schema every five minutes or every hour.

Ahmed Tarek: And you need one says that all the Spanish why even success or a failure to notify the schema owners about that automatically so you can build or reschedule as well the schema notification box.

Ahmed Tarek: The last step is the content tab that includes the dashboards that you are creating by yourself, or a T shirt dashboard that people share with you in the stack.

Ahmed Tarek: So, starting from now we'll start to execute the first step, which is connecting to the data issues.

Ahmed Tarek: So here, for example, if I would like to connect to a new data source, like my sequel data source, for example.

Ahmed Tarek: I think like noon, I can go for advocacy source and I can go for my sequel directly once I click I can name my data source.

Ahmed Tarek: And I can put the required credentials from the user perspective and the password as well, and the connection string from the host or the IP the hostname or the IP from the both number, and finally the database name.

Ahmed Tarek: For the lab when to use an existing connection for my sequel that includes baker for a lot sales data for online store.

Ahmed Tarek: So you can find that on putting the required credentials as well, and you can find that as well this data source will be ready for you in your cloud cluster as well.

Ahmed Tarek: So we have this existing connection, what I will do now is testing the connection to make the connection is successful between important and does the distribution.

Ahmed Tarek: So this is the first step, I connected now to the mysql database that might as well as you can upload multiple files, if you would like like csv excel text files.

Ahmed Tarek: And you can go for the liquor destination, if you would like to submit your vehicles of Matthew insights to a specific to this destination for ftp Google drive Google sheets one drive or through ftp as well.

Ahmed Tarek: The second step after creating or connecting to the required data source is creating the schema and such.

Ahmed Tarek: So I will start here from, then you part.

Ahmed Tarek: I will find different options, I will select the schema where the box.

Ahmed Tarek: So I will name the schema.

Ahmed Tarek: My store.

Ahmed Tarek: I will select the data source which is online store and you can put a description as an optional for your.

Ahmed Tarek: For your schema itself, so I can put it here since schema.

Ahmed Tarek: After that you can go to black next.

Ahmed Tarek: If you remember in the architecture, we are talking now about the data acquisition component and you can find that, if you remember, as well as the schema inspection or the schema detection box now, I can see the database itself here.

Ahmed Tarek: So once I click on the database name a can find the underlying peoples that I have access to and in code that is needs read only access.

Ahmed Tarek: So while you are start to click on a specific table.

Ahmed Tarek: Here so what's happening here directly import identifying the structure of the table itself from the column names, the source type for each column.

Ahmed Tarek: The in kota label which is by default it's the same, and you have the ability to change it from there, or you can change later if you would like, and the encode that mapping type as well.

Ahmed Tarek: directly and they put the function it's a primary key or it's a dimension or it's a metric itself.

Ahmed Tarek: And, as well as you have the control to select a specific people's to be loaded or to be mapped into equal to engine.

Ahmed Tarek: Besides, that you have the ability, as well to control the Van needed the columns that you need to print, so you can select or the select specific columns to bring it into input.

Ahmed Tarek: And for each table here that you selected, you can see here a customized sequel box.

Ahmed Tarek: So you can find here to select the statement that you've got that is using for each table to adjust the date itself.

Ahmed Tarek: So this is a select statement that important values while start doing the loan limits.

Ahmed Tarek: And if you would like to get execute to make sure that the connection is successful and the data sources is retrieving data before creating the schema inside in code itself.

Ahmed Tarek: As well as if you have a very customized the query and you need to bring have to put this customized liquidity to bring the output inside the quarter.

Ahmed Tarek: You can use the custom sequel table here to put your query that you can run on your source system and the output will be saved the schema the output itself on the structure of the object itself will be saved inside the quarter, then the data will be used as well to report itself.

Ahmed Tarek: For the lab here, we will select all the tables so we'll see here a selected is 32 tables from info to spot.

Ahmed Tarek: offer that I will click next.

Ahmed Tarek: And here, you can find that in kota by default can detect.

Ahmed Tarek: The existing relationship that you have in your system if you have a foreign keys identified or created a new system, so I will keep it as default.

Ahmed Tarek: Then I will carry the schema.

Ahmed Tarek: So now, the schema is kitty, and this is the schema beach that you can see here then now I don't have any data loaded into the product it to just the schema itself and that people's itself, so you can see here the tables less from the tables from the table stop here, you can find that.

Ahmed Tarek: At the confined for each table type the data source, the number of columns as well, and the number of related to understand the number of rows is the you.

Ahmed Tarek: As well as again point the year the dual stack so the list of domains, you can find the list of joints and during the dates from the during tap here, you can search for specific doing if you would like, as well.

Ahmed Tarek: And here, you can see, or go for a specific jury and the goal for the join details itself, if you would like to do edit over that or even remove that way, so you have full control over the joy that we are bringing or map directly from your source system.

Ahmed Tarek: And we have the diagram you.

Ahmed Tarek: That giving you.

Ahmed Tarek: A ui about how each table inside in code a link to get based on the mapping that in code that that for the doing that already happened before.

Ahmed Tarek: So if I go, for example, for a sales order detail I collect here, you can find some kind of information from the number of columns the number of related joins the now still we don't have rows.

Ahmed Tarek: Later, you will see the data for the size for that table on the desk on the desk itself and the key columns that I have in that table, and there is the doing the things that I have here.

Ahmed Tarek: And if I go here and they collect, for example, for this one this bottom beside the table name.

Ahmed Tarek: It will open me another screen for the table details inside the table itself.

Ahmed Tarek: So you can find the same structure from the labeling pot the column label and you can change it if you would like, and the column type and the function, you can put a description, if you like, for each column, you can do infection as well.

Ahmed Tarek: And here we have something called a foreigner columns he would like to get a revenue column or something like that you can do that as well, using the formula builder and will show that show that you in details later in in the lap.

Ahmed Tarek: So I can go here for the data sets to find the data source that you're picking from the online store the mysql one, and you can see, see also the during staff for that table that includes the related joins for this table.

Ahmed Tarek: And you can see the filters for that you can apply for that deep, we have two different types of the filters.

Ahmed Tarek: The first one is the runtime security filter, if you remember, if you would like to apply our living security.

Ahmed Tarek: For a specific rules based on different users or based on a specific security roles for each user that will love to the system, you can use the runtime security filter to control which users can see which data.

Ahmed Tarek: And the last filter it's a logical filter if you would like to put, for example, a filter to just load the data for the last 10 years and instead of load the old data, so you can also put the road filter that you need here inside the physical stuff for each day.

Ahmed Tarek: I will go back for the schema CAP for the schema of my store page.

Ahmed Tarek: Then, after going to the details that we see right now, now we need to load the date so I have here 32 tables with 33 doings identified, and we see that, from the back from page itself.

Ahmed Tarek: So now, I will do the loop, I will do the full loop, because this is the first time that i'm loading the data so i'm loading all the data from the source system into encode the storage which important will store the data into the park a format over the US.

Ahmed Tarek: So I will start the full was home here.

Ahmed Tarek: So we have another different load time, which is the incremental loop, so you need every five minutes to bring the new records of the change that happened.

Ahmed Tarek: Across the data, a new tables when you only use schema can also schedule or applied the incremental load by applying for a while increment the logic that you need.

Ahmed Tarek: So why the extraction is happening, you can see here the loading time counter and you can see from here, the last load status.

Ahmed Tarek: If I can click here on the actual traction part you can find the BP page of this load job or the schema right now, and the stage of.

Ahmed Tarek: That attempt extraction the load for each table it's the low time start time duration right now for extraction and salute as well.

Ahmed Tarek: And you can see, as well the rejected tools, if there are rejected roads be for a specific customer problems or data types problems or anything you can find the rejected rose as well.

Ahmed Tarek: And if there is a failure happened, while doing the load, you can see, for each table, what did that exact error or the reason for failing that, that table as well.

Ahmed Tarek: So did you refresh here, so you can find that everything from a status perspective is successfully load the world peace.

Ahmed Tarek: So if I go back again to the schema here.

Ahmed Tarek: So you can find here that we did the load for 25 million records loaded from this mysql database into important.

Ahmed Tarek: In one minute, and you can find the data size for this data source over the desk it's around 700 megabytes.

Ahmed Tarek: Although the original size for this database database around 10 and up, and this is what we were about regarding the podcast storage and the compression that the key is doing, while extracting and storing that data into a calmer format and doing the deed application itself.

Ahmed Tarek: So now everything is ready, from the connection part from the schema creation part and loading the data into in code.

Ahmed Tarek: So let's jump for a specific table, for example, bill of materials.

Ahmed Tarek: So you can find here a bottom that's called a preview beta if you would like the previews that they that that's loaded into import.

Ahmed Tarek: So now i'm previewing the beta here, so I can find you a sample of 100 records only from the total number of records here so and you can find the year for each column, that you can see, the minimum value the maximum value and the number of months for for each column.

Ahmed Tarek: So you need to make sure that the data is loaded successfully and no problem from the vehicle or the perspective.

Ahmed Tarek: So from now, I can start to build and insights from the content tab or even from the schema CAP here to start and doing exclude.

Ahmed Tarek: But let me start from the content that so now that the schema is created and data is loaded I will go for the content tab to start to build the first visual and the first inside inside that.

Ahmed Tarek: So I will click you.

Ahmed Tarek: I can add multiple folders to include multiple vegetable inside each folder to organize my dashboards as work, I will create here, and you that I will name this, therefore, as a sales.

Ahmed Tarek: overview.

Ahmed Tarek: I can put a description as well if I, I would like, and I will click act.

Ahmed Tarek: Now, I can see that I can add my first instinct.

Ahmed Tarek: So I will click on add inside.

Ahmed Tarek: I can see here, this is the analyzers pre to create the fittest inside.

Ahmed Tarek: So if we can start the from the Left, I should here manage my data set and select the tables and the schema that I need to build inside of top of that.

Ahmed Tarek: So I go for the manager of the data set I will go for tables.

Ahmed Tarek: I will select the schema that I can eat it right which is my strength.

Ahmed Tarek: I will close that screen.

Ahmed Tarek: That, I can see here, I can bring that to collapse them.

Ahmed Tarek: laughs I have the option to collapse on the schema level and the table levels.

Ahmed Tarek: So I club, so that I can see all the schemas right now here on the left.

Ahmed Tarek: And from that side you can it shows that require visualization that you need visualization type that you need so important is providing different visualization tax form tables from a charts from kpis as well, and each tax as well.

Ahmed Tarek: So I will choose here that donut troubles.

Ahmed Tarek: And I will search for line today.

Ahmed Tarek: So this is the light Bolton, I will drag that into the measured part.

Ahmed Tarek: And I can see here the column details as well to see sample data, the column name the data type as well, and the function for that one, and this is the source of the call where this column is coming from.

Ahmed Tarek: Then I will look for a dimension which is the category.

Ahmed Tarek: So I can move forward second ticket in him as well and see here the same for the sample data and the column details.

Ahmed Tarek: I will bring back particularly name from here.

Ahmed Tarek: So now, I can see all of that is happening on the fly to aggregates, and to do the some for the line total.

Ahmed Tarek: Based on or grouping by the category name.

Ahmed Tarek: And if I can say that something if you clicked for the measure itself, which is aligned to this URL.

Ahmed Tarek: and go down in this box, you can find something called a query plan.

Ahmed Tarek: If you click on the creative plan you can find here the path that the engine went through to generate that inside, if you remember as well, when we talk about the direct data mapping we mentioned the smart we routing part, so this is a ui of how the engine is.

Ahmed Tarek: Presenting or executing they weren't really plan to generate that inside.

Ahmed Tarek: Now I will put a quick name for that insight as a title, so I can name, it seems by.

Ahmed Tarek: degree.

Ahmed Tarek: And I will click save.

Ahmed Tarek: And another save to save that insight.

Ahmed Tarek: So now, the first insights has been successfully created on the sales overview dashboard.

Ahmed Tarek: But if you can see, while I start to create that inside and click on manage data set to select the schemas.

Ahmed Tarek: I see a lot of tables, I see the 32 tables actually and a lot of columns on the left side.

Ahmed Tarek: So if you are talking about more complex data sources such as earpiece so you have Samsung of tables and cells and columns in that source systems.

Ahmed Tarek: So if we can provide the business was a simplified view or a semantically to bring specific columns in one view, I think this is better for the business to achieve the self service or not it's as well.

Ahmed Tarek: How we can do that, so I will start now to build a business escape.

Ahmed Tarek: So I will go for the businesses schema tab from here.

Ahmed Tarek: click on you.

Ahmed Tarek: I will create a business schema.

Ahmed Tarek: I will name it.

Ahmed Tarek: Business a schema for sales.

Ahmed Tarek: I will click save.

Ahmed Tarek: Then inside the same businesses scheme, and you can add multiple views so beautiful the sales, if you for the HR view for the marketing inside the same business the schema a few black.

Ahmed Tarek: I will create here a one view.

Ahmed Tarek: I will name it for you to school sales.

Ahmed Tarek: And I will click the plus bottom from the left here.

Ahmed Tarek: to choose that schema the physical schema that I need to bring the calls from.

Ahmed Tarek: I will click on mice to.

Ahmed Tarek: That I will keep them.

Ahmed Tarek: Here I will search for specific columns like customer ID.

Ahmed Tarek: From that part.

Ahmed Tarek: I will search for like total.

Ahmed Tarek: And I will do something for like total I will name it as revenue.

Ahmed Tarek: To be appear to the business users as a revenue model and total.

Ahmed Tarek: And I will pray that particularly port.

Ahmed Tarek: And I will name it, as well as particularly only.

Ahmed Tarek: I want to bring the subcategory.

Ahmed Tarek: So i'm searching for that.

Ahmed Tarek: subcategory name and I will photo as subjectivity as well.

Ahmed Tarek: And I would like to build two formulas.

Ahmed Tarek: The first formula that I need to build is regarding the product how I can build that on top of the business is key.

Ahmed Tarek: So I can you can see from the Left as well here, and you formula.


Ahmed Tarek: And this is the legal name I will name it, as well as profit.

Ahmed Tarek: And they kind of choose which is a dimension or measures it's a measure, and then I can set the formula itself, while I clicked on set the formula.

Ahmed Tarek: I can find on the right side, all the supported function that include the hats and actually this kind of excel like for me it's very similar for the excel itself so if you collect, for whatever.

Ahmed Tarek: formula, you can find the kind of fun help and that example to give you about somebody about the function and the example how you can run this formula and how we can write this full moon right right.

Ahmed Tarek: Right now, I will bring that revenue to bring sorry that profit part, so I will bring the line filter from here.

Ahmed Tarek: minus.

Ahmed Tarek: opened a packet.

Ahmed Tarek: pink standard costs.

Ahmed Tarek: As well Double Click on the column name between just put it here inside the food itself just a Double Click.

Ahmed Tarek: multiplied by.

Ahmed Tarek: The quantity.

Ahmed Tarek: Be older.

Ahmed Tarek: quantity.

Ahmed Tarek: This one from the seeds on the details.

Ahmed Tarek: Here can Double Click on that.

Ahmed Tarek: So I just put this simple formula to calculate direct the profit, so the revenues minus the cost multiply by the order quantity.

Ahmed Tarek: Then I will click on validate and safe to make sure that i'm betting from the sentence perspective, the Prophet interact way of the formula invite way.

Ahmed Tarek: Great so the other one of the other format that I need to build is regarding that date, I will I need to bring the month start date for each of the date that I have the month start date.

Ahmed Tarek: So I will get it another formula in the same way to bring the new formula.

Ahmed Tarek: I will name it here older, they will without any spaces, while you are putting the name, you can put.

Ahmed Tarek: underscore something like that, but in the label it's a free for you to put any spaces, or something like that, so I will put here older space they.

Ahmed Tarek: And I will set the for linear again.

Ahmed Tarek: By bringing.

Ahmed Tarek: A function, which is called the month thought the if you can find it.

Ahmed Tarek: And I wouldn't put here.

Ahmed Tarek: But it was the column, which is older.

Ahmed Tarek: They.

Ahmed Tarek: promising school.

Ahmed Tarek: Then I will click on validate and save as well.

Ahmed Tarek: So I created two formulas and you can find these two formulas as well inside.

Ahmed Tarek: The lab guy that the T shirt with you over the strap as well if you mess, how we can put the form of what's exactly that the form itself.

Ahmed Tarek: So I created that view by printing if you can see here i'm painting different columns from the tables in one view and the kind of labeling for that columns for the business understand.

Ahmed Tarek: Then I would click that.

Ahmed Tarek: So now, the business view or the businesses schema it's successfully carried by just drag and drop there quietly.

Ahmed Tarek: And, and any point of time, if you would like to go back for this business is schema and add any new business question was from I can say.

Ahmed Tarek: And this business question or exist ad hoc report require you to add another column or another formula so easily you can go engaged for this, this is the schema go for the edit and drag and the drop as well the required, then he columns or building information, if you would like.

Ahmed Tarek: So I can click here over zacks upload data to accept lower data for this business.

Ahmed Tarek: I will click here on explore the.

Ahmed Tarek: Once I accept load the data it's open automatically to the analyzer screen and on the left here a can't find the name of the businesses schema.

Ahmed Tarek: And the name of the business view that I created under this business schema and the columns that I drag and drop and rename them into a business and the business can see the final or the label itself.

Ahmed Tarek: And also, I can see a sample data as well, and they can see for the customer ideas well assemble data, he would like to see songs like that.

Ahmed Tarek: And I will go for from the measured name from sorry from the business view name itself which one this one view sales, I will drag that to the measure tray.

Ahmed Tarek: So and used a lasting table by default so just to give me a way to explore the beta and see that everything seems fine from the perspective itself or from the data perspective itself.

Ahmed Tarek: By dragging that business view name into the measure of trades.

Ahmed Tarek: So now, I will exit that after exploring and make sure that everything is going fine and going back again to create or to continue on creating multiple insights into the sales overview dashboard that we created a donut chart from there.

Ahmed Tarek: So this is is the content tab again.

Ahmed Tarek: This is the sales overview.

Ahmed Tarek: This is a donut chart that we get a right.

Ahmed Tarek: I will click add here again from the from the right side to add a new insights and I will open the analyzers P ugly Now I will ignore the my store schema from the table, I will be selected that.

Ahmed Tarek: and go for the reviews tab so just praying that acquire the data.

Ahmed Tarek: From the business is key, but I really like that you can see here right.

Ahmed Tarek: And I will go to create the the dominant the chart that the right way.

Ahmed Tarek: I will create either gonna try again.

Ahmed Tarek: So praying just.

Ahmed Tarek: The category and the revenue as well.

Ahmed Tarek: So it's the same thing from the physical schema and from the businesses schema but, and you can find as well, to see the same really blank.

Ahmed Tarek: To understand what's happening here how I can have this inside integrated, but actually now it's more simplified the end it's more related to the self service analytics instead of seeing a lot of tables a lot of columns on the left side you just see the part of the business view itself.

Ahmed Tarek: So let's create.

Ahmed Tarek: A line chart, for example.

Ahmed Tarek: So I will go here to select a line of chart.

Ahmed Tarek: As well remove I can take it from here.

Ahmed Tarek: And the choose another one, which is the old the dates.

Ahmed Tarek: Already go here to drink the order date on the grouping dimension, you remember that this order date is a Formula One could eat right.

Ahmed Tarek: So you can see here that I put the order date along the Web revenue and if I go again to check media plan.

Ahmed Tarek: To be changed it writes automatically.

Ahmed Tarek: Now I need to do a basic filter over the inside here.

Ahmed Tarek: So what I will do as well.

Ahmed Tarek: Put to drag the order bait on the individual filter box.

Ahmed Tarek: And what we need to print here to choose operated can find the from that scream the operator for the felt I needed to be greater than or equal.

Ahmed Tarek: And after that you can find the less I need to be greater than or equal fairest of we need one.

Ahmed Tarek: So the filter it's happened automatic why I click and select require the operator and the wire event as well.

Ahmed Tarek: And I would like to complicate things more than that, to put a decade typically in the coloring dimension, so I will drag like a typically in the coloring tray as well.

Ahmed Tarek: So you can find the that I created a caloric dimension with the category part as well.

Ahmed Tarek: Just wait for five seconds or 10 seconds to make sure that we are on the track.

Ahmed Tarek: So I will leave that insight.

Ahmed Tarek: As an apiary.

Ahmed Tarek: revenue.

Ahmed Tarek: over time.

Ahmed Tarek: And I will click save.

Ahmed Tarek: So now, I have two insights in one dashboard in one tab right.

Ahmed Tarek: So I would like to add kpis like a seminary kpis, so I will do that again from scratch, if you mess something how I can get that insight, so I will go here to click add again.

Ahmed Tarek: manage that intersect I will move this one and go for.

Ahmed Tarek: be a sales for the manager to support.

Ahmed Tarek: A wallet choose from here the visualization type, which is KPI.

Ahmed Tarek: I will bring revenue.

Ahmed Tarek: In the measure tray.

Ahmed Tarek: OK, I can select here the number format which I would like, I can say dollar rounded.

Ahmed Tarek: like that.

Ahmed Tarek: Then.

Ahmed Tarek: I can bring as well.

Ahmed Tarek: Another KPI, which is the prophets.

Ahmed Tarek: me this one and bring the profit, as well as in the dimension here the measure trace or here and I will also achieved the number format to be.

Ahmed Tarek: Under the ground as well.

Ahmed Tarek: Then I will pray the customer ID.

Ahmed Tarek: Here in the message today as well.

Ahmed Tarek: And I need to change the aggregation function for the customer right I don't need the default, one which is the sum I need another one so from this screen, I can see, I would like to count the number of distinct customers that they are buying my products.

Ahmed Tarek: I will click on the States.

Ahmed Tarek: So now, and I need to rename that one to the label for that number of customers.

Ahmed Tarek: So you can see here and I need to name that as an kpis.

Ahmed Tarek: And click save.

Ahmed Tarek: After that I will click on save.

Ahmed Tarek: So now, you can see the multiple stream visualization types that we created inside the same vegetable and inside the same time.

Ahmed Tarek: So what we can do is.

Ahmed Tarek: If I would like to edit the layout itself.

Ahmed Tarek: So I can go for here from the three dots here, if you would like to again to can drag and drop and Edit your layout or to personalize out, you can do that, and even if.

Ahmed Tarek: If I would like to go for this one for the KPI I need to bring this KPI on top, I can go directly from here to bring it.

Ahmed Tarek: and bring it again up from here, and again have to be on the POP.

Ahmed Tarek: Then I can click on this line of chart.

Ahmed Tarek: To bring it up beside this one.

Ahmed Tarek: And I needed to be on the right side, for example.

Ahmed Tarek: And I need to collect now safe to update required laid out again how I can do that just I can stop on one of these insights that I have here from this part they can see the hand, so I can just move it to anywhere that I need.

Ahmed Tarek: Once I did that I will see that screen appear that they can use also the controls do that to move down to the bottom right or moving left as well.

Ahmed Tarek: And after that can cancel have safe to update your layout as well.

Ahmed Tarek: I would like to get heat last.

Ahmed Tarek: Inside with you.

Ahmed Tarek: Which is an aggregated table and we'll try to get the heat up a conditional for backing on top of that.

Ahmed Tarek: yeah so let me bring or create a new aggregated table, so, as usual, I will go for manage data set again go for the use of the sales.

Ahmed Tarek: And the removal selection for this one, because yeah OK.

Ahmed Tarek: Then, after that I will select aggregated table from here.

Ahmed Tarek: I will bring the category and subcategory in the group and dimension.

Ahmed Tarek: If they agree.

Ahmed Tarek: And subcategory.

Ahmed Tarek: And the profit and the measure year.

Ahmed Tarek: We can set the profit here.

Ahmed Tarek: $2.

Ahmed Tarek: Products.

Ahmed Tarek: And we can also pre revenue port beside the Prophet.

Ahmed Tarek: And format that into dollars.

Ahmed Tarek: So I have now an aggregated table right So if I would like to give you a conditional formatting over the prophets.

Ahmed Tarek: So I will go there here from that screen to click on the profit.

Ahmed Tarek: You can find here.

Ahmed Tarek: And this one add a conditional formatting.

Ahmed Tarek: Once you think you need to set up another screen open to select the operator itself, so I select the operator right now to be greater than greater than the value 1,000,003 66 seals and I will change the background for that sells applied this additional formatting to green.

Ahmed Tarek: I will add another one.

Ahmed Tarek: Greater that video.

Ahmed Tarek: I will keep it that as default I needed to be on an orange.

Ahmed Tarek: The last one.

Ahmed Tarek: I want to change here the operator to be less than view as the negative one to be an indirect.

Ahmed Tarek: And we'll just change this one, the sequence, I mean yeah So you can see here the Prophet the last column apply to it very wire conditional formatting.

Ahmed Tarek: And I can lead back here.

Ahmed Tarek: Product.

Ahmed Tarek: He paints.

Ahmed Tarek: and collect sing.

Ahmed Tarek: So with that layout that I need so now, I can say congratulations that you created the first dashboard and created the full pipeline from the data sources to the dashboard level like that you can see, on the screen.

Ahmed Tarek: But I would like also to show you some of other capabilities that we can see as well, for example, if I would like to collect for bikes here.

Ahmed Tarek: So once I clicked for a specific thing specific dimension that I have here a specific measure that will there will be felt.

Ahmed Tarek: Because they are connected TVs right, so it can see here, these numbers are changing so I can understand, for the most continually sales, which is.

Ahmed Tarek: The bikes what's happening inside that's where all the drops that they have, for example, inside the box itself and the table itself has been felt for that think of that view.

Ahmed Tarek: If I would like, as well.

Ahmed Tarek: To add another filter, for example, for a subcategory so I can click from the sabbatical itself, so the wall dashboard will be felt again for this opportunity, so if.

Ahmed Tarek: I will always that usually away a file on opening vegetable every time no need every time to go and do the filters, again, I can save that filters.

Ahmed Tarek: As a bookmark so I came over here to click on the bookmark side.

Ahmed Tarek: I would like to add a bookmark right and I will leave that bookmark to name it, for example, bikes.

Ahmed Tarek: It can be public, if you would like, for every user or it can be only for you, and they can click out.

Ahmed Tarek: So now, you can find that I have bikes as an.

Ahmed Tarek: bookmark for the filters that i'm usually using so I can remove it, if I would like, and if I would like to apply again I can apply once just to click on that.

Ahmed Tarek: I will create them all.

Ahmed Tarek: The other thing that I can create a personalized view as well if I would like to create a personalized view I can do that.

Ahmed Tarek: So, for example, from here, from this one, you can find this icon the second icon from the top, right here it's called the personal lives, I will click on that I will click on personalize jelly out that I can see.

Ahmed Tarek: You have the full control to select a specific or to the select a specific insights so, for example, I don't for my personalized view I don't to see the product updates.

Ahmed Tarek: Once I clicked, for example, for this one was inside, you can actually control the layout as well, if you would like to change the layout and we immediately output such and the insight settings itself so, for example, for this one.

Ahmed Tarek: For this insightful example, they would like to change any settings for for any of the insights you can do that as well, so now, I have a personalized view.

Ahmed Tarek: That you can see from there, and I will click save.

Ahmed Tarek: After everything that he out settings.

Ahmed Tarek: So this now my personalize the view, and I can go from here to go back to reset to.

Ahmed Tarek: The origin is.

Ahmed Tarek: So I can and can have a switch between them, I can go a need for the personalized view so you can go for new per slide do or getting back to your original view as well.

Ahmed Tarek: So after creating the dashboard itself if you go back here sorry again for the same overview and go for the original one or for the personalized one if I would like let's go for original one.

Ahmed Tarek: So I can share that that would words very wide the stakeholders from here, I can share the access and even I can send a schedule on Skype will.

Ahmed Tarek: be sending this dashboard for specific users with a specific format as well.

Ahmed Tarek: So now I clicked on send over schedule, so you can choose here your groups that you need to send to see your CC or bcc even and you can search here for that.

Ahmed Tarek: admin, for example, I have a group here it's called admin you can treat that here.

Ahmed Tarek: And you can put your subject selected tags, if you have multiple tasks in one dashboard and you like to share only specific tasks.

Ahmed Tarek: And here, you can you can change the data format needed to be sure it's an html or html in the body itself or file or csv or excel for the tables inside that we have all the old days old to be.

Ahmed Tarek: exported as a PDF and you can put actually a body for the stakeholder and you can choose from the spiritual part that you need.

Ahmed Tarek: Every day, every hour or at specific time and the start date for the scheduling thought or you can also share it now, with specific users or something like that, so I.

Ahmed Tarek: select here the admin or search for the super admin, you have the control for the shared one, to make sure they can view, only the dashboard you can share it again, or the can edit this move as well.

Ahmed Tarek: So that's the whole journey that giving you a brief how you can't create the old pipeline from the data connection to the schema design itself.

Ahmed Tarek: Can you think a semantically a bubble that which is the business schema inside equal.

Ahmed Tarek: And that content tap to create a full batch was multiple insights and easy way and scheduling Bart too skinny will you sharing their wounds scheduling your schema loads or the data notification or the beta kpis but.

Ahmed Tarek: Let me have a week luck for the Q amp a that we have.

Ahmed Tarek: yeah yeah I think we covered the part of the personalized one, I think, Martin David answered most of the questions, thank you, Martin baby from that.

Ahmed Tarek: yeah, so I think the last thing that I would like to show you and we talked about that, within the architectural.

Ahmed Tarek: If you go back for the schema tab and going back to my store.

Ahmed Tarek: And I would like to create a new.

Ahmed Tarek: materialize if you, if you remember the part of the math realize that your answer data enrichment component box.

Ahmed Tarek: So I will click on naturalized view here, you can choose that require language or the English language that you need to write through the scraps from the options that we mentioned before.

Ahmed Tarek: And you can see here edit in a notebook, which is the zipline notebook that most of the science like just the moon books or something like that, so you can start to write like that and upload will be saved and then separate table.

Ahmed Tarek: in sight in code and get the link again.

Ahmed Tarek: So if I go back to the presentation again.

Ahmed Tarek: So now we we know in code and belt the default from end to end.

Ahmed Tarek: The last thing that I would like to mention is the Community the Code, the has its own community community, you can go directly and do a registration and creative incorporated calm.

Ahmed Tarek: It includes a lot of useful information for you from the announcement bar from the events upcoming events from a knowledge base as well, you can share you ideas with us from a features, or something like that.

Ahmed Tarek: You can see a lot of discussions with info that you users and you can ask good questions and because the experts will answer that to you as well over the Community part.

Ahmed Tarek: And finally, thank you so much, again, for the time for attending that lab and we are planning to have multiple following sessions to cover more advanced topics, this is, we can emit a fundamental one.

Ahmed Tarek: How we can use the code to the data or data pipeline after that well well two more things are more advanced topics to cover another areas that you can use in code to them.

Ahmed Tarek: They could get so much and anytime that you can share your questions on top on the Community, and we can help you and you can create also use cloud clusters and you have anything, please don't hesitate to contact us as well, thank you so much, and enjoy the rest of your day, thank you.