This article is a step-by-step guide to assembling and publishing a small, open-source Python package. While not all of the steps below will be appropriate or desirable for every package, each of these can contribute to the accessibility and maintainability of the package. Note that this material is meant to serve as a roadmap and overview; it is not a thorough review of all the nuances and trade-offs involved. You are encouraged to consult additional resources (links are included throughout the text) for other options and viewpoints on how to approach every portion of the process.

Each section covers a task or category of tasks related to organizing and publishing a package and, where appropriate, provides templates and examples. Note that depending on when you are reading this article, some of the steps, templates, and examples may be out of date. …

There are a variety of obstacles that can make it a challenge to deliver the benefits of privacy-preserving secure computation capabilities to organizations. These can include a lengthy process for obtaining approval to install new software, the cost and personnel effort associated with provisioning new cloud resources, and the technical hurdles involved in integrating existing infrastructure with new APIs. Fortunately, there is a way to sidestep these challenges for at least some secure computation workflows by delivering them via a web browser.

Privacy-Preserving Matching and Computation

Facebook AI Research (FAIR) recently released an open-source framework that makes it possible for two parties to match their respective data sets based on a common identifier — and to perform some aggregate computations over the results of that match — without requiring that either party share data with the other party. The framework is implemented in Rust, with Facebook’s stated rationale for the choice including “superior safety features and ease of writing multithreaded code”. …

Suppose you are implementing a service that works with data sets that represent routes on a map (e.g., collections of driving directions or logs of past trips). The road network is represented as a graph with nodes and edges, and each route is conceptually a collection of edges. You are tasked with choosing an appropriate data structure for routes that meets at least the following criteria:

• it should be possible to deduplicate large collections of routes,
• it should be possible to use individual routes as keys into a dictionary (e.g., …

If you are working on a project in which you must enumerate, traverse, and/or test every possible combination of elements from one or more finite (or even infinite) collections, Python provides a variety of ways to do so. This article reviews some of the syntactically concise ways to do so, while also addressing some relevant memory utilization aspects. In particular, the focus is on comprehension syntax as foundational building block that can be employed in conjunction with functions and recursion.

Conventions for Terminology and Notation

In order to maintain consistency across examples and to keep outputs deterministic, this articles follows the conventions enumerated below.

• Python lists are used to represent both collections that may have duplicates and sets that do not have duplicates. For example, the set {0, 1, 2} is represented using [0, 1, 2]. …

By building and deploying web services or data analysis workflows that employ emerging privacy-enhancing and secure computation technologies such as secure multi-party computation (MPC), organizations can provide new services, identify and leverage new business opportunities, reduce risks and costs for both themselves and their customers, and comply with evolving regulations. MPC is already being incorporated into some industry, government, and nonprofit software solutions and workflows. But leveraging its advantages to address an organization’s data privacy challenges requires an understanding of how its features can satisfy technical, business, and legal constraints. …

Suppose your Python library needs to load some sensitive binary data from a file into a contiguous block of memory (e.g., in order to use it for some application-specific operation). Furthermore, you have some additional requirements that must be met for security and auditing purposes:

• you need to ensure that your code does not inadvertently cause the interpreter to copy any part of the data loaded from the file into some other region of memory,
• you need to log the memory address at which the data was stored, and
• you need to clear the memory region that held the data by overwriting it with random bytes. …

Suppose you are maintaining a domain-specific machine learning library. Users of the library’s API expect that every machine learning algorithm offered by the API will have the same interface (i.e., the same methods with the same signatures) regardless of its underlying implementation. You would like to allow a community of contributors to define new algorithms that can be added to the library, but you would like to reduce your own effort and that of contributors when it comes to validating that a new algorithm conforms to the API.

Python metaclasses are the underlying, higher-order constructs that instantiate class definitions. Understanding what metaclasses are and how they can be used gives you a significant amount of control over what happens when a new class is introduced by users. This in turn allows you to constrain users when necessary and to provide assistance to users that can save them time and effort. …

Suppose you are working on a project in Python that takes advantage of Python’s support for higher-order functions. Perhaps you are allowing users to specify their own hooks or event handlers within a web application framework, or you are creating a unit testing library that generates test cases for a user-supplied function. In many such use cases, you might find yourself dealing with two related scenarios:

• user-supplied functions can return results of different types and you do not know in advance what the result type of any given user-supplied function will be, or
• your code needs to generate inputs to user-supplied functions that have the type the user-supplied function expects. …