May 2021 Summaries

Pandas is a powerful Python library for data manipulation and analysis, offering numerous options to read data into DataFrames. This guide explores some of the most useful methods for loading data from various sources such as CSV, JSON, Parquet files, SQL databases, HTML documents, and more. It also covers reading data from remote storage solutions like S3, Google Cloud, SFTP, or GitHub using FSSPEC library. Pandas can automatically detect compression algorithms and decompress the data before reading it. The guide provides examples of reading different file formats and working with remote files, highlighting the flexibility and ease of use of Pandas for data processing and visualization.

Setting up a deep learning environment with TensorFlow 2.4 and GPU support can be streamlined by following a series of steps, particularly for cloud VMs running Debian or Ubuntu. The process begins with the installation of the Anaconda package manager, followed by creating a Conda virtual environment to manage Python 3.8 installations. A crucial step involves installing NVIDIA's CUDA and cuDNN developer libraries, as TensorFlow versions are specifically compiled to utilize certain versions of these libraries for GPU acceleration. Despite the lack of a Conda install script for TensorFlow 2.4, detailed instructions from the TensorFlow team can guide users through manual installation. The setup concludes with a new method provided by TensorFlow 2.4 to verify GPU availability, ensuring that the environment is properly configured to leverage GPU capabilities for deep learning tasks.

Gretel is working on creating an API-driven approach to privacy engineering, backed by Gretel Cloud. This involves a configuration-driven approach where users can construct their privacy workloads in configuration files that can be sent to the REST API. The company has spent time with customers helping create optimal synthetic data configurations based on the type of training data being used. They have found some good synthetic data configurations and are making them available as templates that can be consumed and edited just before training a model. These templates will be useful for users regardless of their data type, allowing them to pick the right parameters needed to train their synthetic data models.