Autoscale for Azure HDInsight is now generally available across all regions for Apache Spark and Hadoop workloads. This feature makes it possible to operate your big data analytics workloads in a more cost-efficient and productive way, so you can drive higher use of your HDInsight clusters and pay only for what you need.
A new Azure HDInsight capability enables you to customize and upgrade your own Azure SQL Database for Apache Ambari during cluster creation.
Ambari is used to monitor HDInsight clusters, make configuration changes, and store cluster management information as well as job history. HDInsight provides a default SQL Database for each cluster. Now, scale up your Apache Spark and Hadoop jobs on HDInsight clusters by configuring SQL Database settings according to your business growth requirements. It’s also possible to start with a basic database and upgrade later.
Azure CLI support for Azure HDInsight is now generally available. This enables administrators to easily manage their HDInsight clusters using simple Azure CLI commands. Administrators can use Azure CLI to automate HDInsight cluster creation, scale up/down and deletion operations, manage script actions and applications, and to understand the capacity usage.
We are pleased to share that support for HDInsight in Azure CLI is now generally available. The addition of the az hdinsight command group allows you to easily manage your HDInsight clusters using simple commands while taking advantage of all that Azure CLI has to offer, such as cross-platform support and tab completion.
We are pleased to announce the integration of VSCode Python extension to handle PySpark interactive for HDInsight in VSCode. The Azure HDInsight Tools for VSCode leverages the Python extension to provide enhanced python editing experience, Jupyter Notebook integration and the Notebook like PySpark interactive query.
HDInsight for VSCode continues to boost the experience of Hive users with self-service exploratory capabilities. The HDInsight explorer in VSCode not only empowers you to browse Hive databases across HDInsight clusters, but also enables you to view Hive table schema and preview data.
The networking resources (such as NICs and load balancers) for new Azure HDInsight clusters created in a virtual network are now provisioned in the same HDInsight cluster resource group.
Previously, these resources were provisioned in the HDInsight virtual network resource group. There is no change to the current running clusters and those clusters created without a virtual network.
This is useful in cleanup scenarios. When you delete the resource group, all necessary resources associated with the HDInsight cluster will be deleted from the same resource group.