pthaine

Data privacy provider Private AI has launched a "privacy layer" for large language models (LLMs) like OpenAI's ChatGPT, called PrivateGPT. The new tool uses AI to redact over 50 types of personal information from user prompts before they are submitted to ChatGPT, replacing them with placeholder data. This comes as OpenAI faces increased scrutiny over its data protection practices, with Italy temporarily banning ChatGPT and Canada's federal privacy commissioner investigating allegations of "the collection, use and disclosure of personal information without consent." Private AI highlights the importance of data anonymization techniques for organizations using LLMs. The company has plans to ensure other companies building on top of generative AI services can protect their customers data using an API they will be launching. Try an example implementation today at chat.private-ai.com

Steps to scale Kubernetes pods based on GPU metrics

Key takeways: (1) ONNXRuntime is the best inference package for Transformer networks; (2) Nvidia Triton, together with ONNXRuntime is the best solution for GPU inference; (3) Optimization matters. It’s quite easy to unlock a >10X performance gain in 2022.

NLP teams: Finding it difficult to keep track of changes to regulations like GDPR, CCPA, POPIA, & LCDP and their implications on how you train your models? Here's a summary of what to watch out for in 2022.

Thanks for your comments! - homomorphic encryption is much much faster now, so the latency argument against won’t hold much longer --> I can see that as being true for private libraries, hopefully this will become reality soon once public libraries integrate GPU or FPGA support

- with multi-key FHE, you could replace MPC, without the integration complexity and increased bandwidth cost

-> Good point! Multi-key FHE is still pretty tricky to implement, however. Though the research is getting there!

- Trusted Execution Environments are not about protecting user data (that’s the purpose of FHE) but rather about protecting the software itself from people having access to the physical machine. An example would be running a sensitive ML model in the cloud: you would want to use FHE to process the user data encrypted, inside an TEE that would protect your model from the cloud vendor.

-> It can be and is used for both code protection and user data protection (e.g., for authentication)

For devs & tech team managers wondering how to integrate privacy into their software pipeline.