ChannelLife Australia logo
Industry insider news for Australia's technology resellers
Story image

No ETL? No worries, with SQL Server 2019's new data virtualisation

By Contributor
Fri 5 Apr 2019
FYI, this story is more than a year old

Article by IT journalist David Williams

Big data is big business, and global software titan Microsoft is making big data its business to streamline businesses with the next release of SQL Server 2019 late this year.

Microsoft states SQL Server 2019’s focus is to make managing a big data environment a lot easier.
In fact, the product supports a rich variety of environments like Hadoop Distributed File System (HDFS), Spark, analytics tools, and even Kubernetes-managed Linux containers.

Previously, Microsoft allowed T-SQL queries inside SQL Server 2016 to pull data from Hadoop and return it in a structured way without moving or copying the data.

In the 2019 release, this concept of data virtualisation is widely expanded to data sources including Oracle, MongoDB, even CSV files.

Data virtualisation means database administrators can virtualise external data in a SQL Server instance, regardless of source, location, and format, so it can be queried like any other set of tables within your SQL Server instance.

Data virtualisation helps you create a single “virtual” layer of data from these disparate sources providing unified data services to support multiple applications and users.

This sounds like a data lake, but unlike a typical data lake there is no need to move data from where it lives when using data virtualisation, and this is a huge advantage over traditional ETL (extract, transform, load) processes that by necessity require delays, extra storage, additional security, and an amount of engineering to setup and maintain.

For database administrators, software developers, data scientists and others this new capability is transformative and powerful in its simplicity.

Quite literally, Microsoft has enabled SQL Server 2019 databases to hold virtual tables which come from some other place on the Internet - whether it is a SQL Server database or otherwise, whether it’s local or far - and query and join and work with these tables just as if they were native to a company’s database.

All complex ETL processes are gone.

There are numerous situations where this delivers tremendous time and functionality savings.

For instance, if a user makes a dashboard showing their company’s revenues by period, office and client out of their main line-of-business app’s database.

Management then asks for budgeted sales figures to be included so the dashboards contrast actuals against the budget.

However, these aren’t included in the main app but instead held in an Excel spreadsheet.

Traditionally, the solution is to manually or periodically upload the spreadsheet into a new table created for this purpose, which your dashboard then queries against.

Or, if they’re using SQL Server 2019, the user merely saves the budget figures as a CSV file on the Finance team’s file share and add this as a virtual table in their app’s database.

The budget figures appear, and can be queried against, and reported on, as if they were always in the database.

Another case: the company needs sales figures to move from its main application into the back-office ERP platform.

Typically, this is achieved by exporting and importing the data between the disparate products.

Or, if someone is using SQL Server 2019, they include relevant tables from the ERP system as virtual tables in the main application’s database, and directly move data via a scheduled stored procedure.

The ERP doesn’t even have to be an SQL Server database, so long as it is accessible.

SQL Server 2019’s data virtualisation supports Cosmos DB, SQL Server, Azure SQL, Oracle, HDFS and DB2.

Other SQL Server 2019 features include

  • transformational insights over structured and unstructured data supporting Hadoop and Spark
  • scalable compute and storage clusters
  • a complete AI platform to train and operationalise R and Python models in SQL Server Machine Learning Services or Spark ML using Azure Data Studio notebooks
  • the choice of programming language and platforms like .NET, PHP, Node.JS, Java, Python, Ruby and more, and deploy the application on Windows, Linux or containers both on-premise and in the cloud
  • real-time analytics on operational data using Hybrid Transactional and Analytical Processing (HTAP)
  • intelligent query processing features that improve scaling of queries
  • security enhancements including Always Encrypted secure enclaves, and Data Discovery and Classification labelling for GDPR
Related stories
Top stories
Story image
Tech job moves
Tech job moves - Adatree, Brother, Databricks, Nutanix & Rubrik
We round up all job appointments from May 20-26, 2022, in one place to keep you updated with the latest from across the tech industries.
Story image
Symbio
Symbio consolidates TNZI business to support APAC expansion
Symbio has recently announced the consolidation of its international business (TNZI) under the Symbio brand to support its Asia Pacific expansion strategy.
Story image
Cybersecurity
4/10 Australian SMEs fallen victim to cyber-attacks since pandemic
Almost four out of teb SMEs in Australia have fallen victim to cyber-attacks since the pandemic began, according to a new study.
Story image
Transport
Third-party automotive apps bear significant privacy risks
Mobile applications for connected cars provide various features to make life easier for motorists, but they can also be a source of risk.
Story image
Sony
Sony launches LinkBuds S - the latest model in the series
Sony says the LinkBuds S will give users a unique sound experience through sensor and spatial sound technology, even in AR games.
Story image
Data Protection
Information management capabilities to meet privacy requirements
Organisations with customers or operations across more than one country face a spate of new and proposed privacy and data protection laws.
Story image
Training
Infosec unveils role-guided cybersecurity training roadmaps
Infosec Skills Roles maps hands-on training and certifications to the 12 most in-demand cybersecurity roles to maximise training efficiency.
Story image
Broadband
Telstra enters into new RSP agreement with Opticomm
Telstra has entered into an RSP agreement with Opticomm (A Uniti Group Limited subsidiary) to provide network fibre services to customers.
Story image
Artificial Intelligence
Frost & Sullivan recognises Genesys as leader in new reports
Frost & Sullivan has recognised Genesys as a leader in the cloud contact centre market for its robust cloud and digital capabilities.
Story image
BYOD / Bring Your Own Device
How zero trust can lead the battle against ransomware
SecOps teams champion a zero trust strategy to support the fight against the escalating risk of cybercrime and help monitor threat actors across a network.
Story image
Cybersecurity
Cybersecurity prompts upgrade for 1.3 billion electricity meters
ABI Research finds Advanced Metering Infrastructure (AMI) and cybersecurity concerns are prompting the upgrade of 1.3 billion electricity meters by 2027.
Story image
Training
Delinea enhances partner experience with new training initiatives
Delinea has announced a range of training and certification initiatives to help scale the capacity of VARs, System Integrators, MSPs and MSSPs to meet the demand for PAM solutions.
Story image
Malware
Fortinet introduces self-learning AI in latest offering
Fortinet is introducing self-learning AI capabilities in its new network detection and response offering, FortiNDR.
Story image
Dynabook
Dynabook refreshes Portégé X30L series with the Portégé X30L-K
The new model contains hybrid-architecture Intel 12th Gen Core P-Series 28W processor options, Wi-Fi 6E, along with Intel Iris Xe graphics.
Story image
Digital Marketing
Similarweb acquires SEO and rank tracking company Rank Ranger
Digital intelligence company Similarweb, which specialises in analysing web traffic, has acquired Rank Ranger, a market leader in SEO and rank tracking.
Story image
Malware
'Alarming' rise in ransomware threats - Verizon report
As criminals look to leverage increasingly sophisticated forms of malware, it is ransomware that continues to prove particularly successful.
Story image
Ransomware
Employees on the frontline of cyber defense - report
In the first quarter of 2022, employees found themselves more than ever at the frontline of cyber defense, according to a new report from Kroll. 
Story image
Silver Peak
The path to an adaptive, modern network
Managing and securing the network looks different than it did just two years ago—especially given that most of these networks are made up of multi-generations of infrastructure stitched together over time.
Story image
Wireless
Hands-on review: HyperX Pulsefire Haste wireless mouse and HyperX Pulsefire XL Mat
With its lightweight Pulsefire Haste wireless mouse and RGB lit Pulsefire XL Mat, HyperX sets out to up your game and add a little colour to your desktop.
Story image
Microsoft
Global cybersecurity insurance market worth $11.5b this year
Future Market Insights finds the cybersecurity insurance market is expected to reach USD$11.5 billion in 2022, growing to $61.2 billion in 10 years.
Story image
Customer experience
Phone, internet complaints increase for first time in five quarters
Phone and internet complaints have increased for the first time in five quarters, according to a new telco report from the Ombudsman.
Story image
Vectra AI
Vectra’s inaugural Partner of the Year Awards revealed
APAC companies Baidam, Firmus, ShellSoft and Macnica have been recognised in Vectra AI's inaugural Partner of the Year Awards.
Story image
Cybersecurity
Comcast to use ThreatQuotient for cybersecurity operations
Comcast, the parent company of NBC Universal and SKY Group, has chosen ThreatQ Platform and ThreatQ Investigations to meet their cybersecurity needs.
Story image
Wireless
Hands-on review: Technics EAH-A800 Noise Cancelling Wireless Headphones
Designed in Osaka, Japan, these headphones just exude quality. They aren’t heavy, but they feel well built and solid.
Story image
Cybersecurity
What every CISO must answer to enable a best-in-class security operations program
It has been widely reported recently that South Australian government employees have been the victims of a cyberattack.
Story image
SaaS
MedCart achieves close to AUD$10 million in sales with help from Marketplacer tech
Australian health and medical start-up MedCart has achieved annual sales of close to AUD$10 million in less than 12 months, utilising Marketplacer marketplace technology to help achieve successful results.
Story image
Phishing
Vishing attacks reach all time high - Agari and PhishLabs
"Hybrid vishing campaigns continue to generate stunning numbers, representing 26.1% of total share in volume so far in 2022."
Story image
Getac
Getac and Optalert expand partnership to promote industrial safety
Getac’s ZX70 Android tablets will be used to support Optalert’s Eagle Industrial early-warning drowsiness detection system for use in mining and transportation.
Story image
Digital Signage
MAXHUB's Digital Signage range to bolster boardroom productivity
The new MAXHUB Digital Signage technology is purpose-built to make every kind of team meeting more effective.
Story image
SpaceX
Australian space tech startup secures SpaceX support
Space Machines Company has secured the support of SpaceX as a launch partner in carrying SMC's Optimus Orbital Transfer Vehicle to space next year.
Story image
Manhattan Associates
Shortening the click-to-customer cycle through smart technologies
Speed of delivery without accuracy is a dealbreaker for consumers. How can retailers operating in an omnichannel environment overcome the challenge of click-to-customer cycle times.
Story image
Surveillance
i-PRO releases smallest AI-based surveillance camera on the market
The new i-PRO mini network camera is now available, with a pocket-sized form factor and full AI analytics functionality.
Story image
Cybersecurity
Asia Pacific plagued by sophisticated bad bots - report
The three most common bot attacks were account takeover, content or price scraping, and scalping to obtain limited-availability items.
Story image
Identity and Access Management
The post-pandemic workforce requires secure IAM capabilities
HID Global discusses what identity and access management means for organisations in today's convoluted digital world.
Story image
Red Sift
Entrust expands strategic partnership with Red Sift
Entrust has expanded its strategic partnership with Red Sift to make it easier for businesses to adopt Brand Indicators for Message Identification (BIMI) standards for email identification and security.
Story image
Ransomware
APAC ranks third-highest region targeted by ransomware
Asia Pacific has ranked the third-highest region globally to be targeted by ransomware, according to cybersecurity firm Group-IB.
Story image
Phishing
WhatsApp and QR codes the next scam threat - report
KnowBe4 has warned it expects to see an increase in QR Codes and the WhatsApp chat platform being used for phishing and other scams. 
Story image
Employment
Tech job moves - Forcepoint, Malwarebytes, SolarWinds & VMware
We round up all job appointments from May 13-20, 2022, in one place to keep you updated with the latest from across the tech industries.
Story image
Phishing
MailGuard warns of new scam targeting Telstra customers
Telstra customers in Australia are being warned of a new scam involving "Unsuccessful Payment" messages.
Story image
Omnichannel
Lexmark launches A/NZ first with retail publishing solution
Lexmark, a global imaging and IoT solutions leader, launched its Publishing Platform for Retail (PPR) in Australia and New Zealand.
Story image
Informatica
Informatica, Oracle enter strategic global cloud partnership
Oracle named Informatica as a preferred partner for enterprise cloud data integration and data governance for data warehouse and lakehouse solutions on OCI. 
Story image
Microsoft
Elevation of Privilege the top 2021 Microsoft vulnerability
BeyondTrust has released its 2022 Microsoft Vulnerabilities Report, finding that Elevation of Privilege is the top vulnerability category for the second consecutive year.
Story image
Microsoft
Microsoft previews Power Platform website design offering
Microsoft has announced the preview of Power Pages, the fifth product in its Power Platform family, designed for low-code makers and professional developers.
Story image
Hybrid Cloud
Barracuda expands cloud-native SASE platform
"The expansion of Barracuda's cloud-native SASE platform for hybrid deployment models and IIoT environments solves a number of challenges."