{"guid":"fb4d780c-f6c0-4669-9fd3-e3e3788d4ca5","title":"From Open Weights to Trusted Models","subtitle":null,"slug":"sps26-110878-from-open-weights-to-truste","link":"https://talks.python-summit.ch/swiss-python-summit-2026/talk/KJJUPP/","description":"APIs of frontier models are the fast way to ship, but rarely the cheapest place to stay. Open weights are a real alternative: you pay for compute instead of per-token rent, the model runs on your own machines so the data never leaves, residency and governance are yours to set.\n\nThe open question is whether an open-weight model is actually good enough for your use case, and which one. A leaderboard won't tell you. It measures someone else's tasks, not yours, and an agent that runs over many turns and calls tools can score well on a benchmark and still fail the real job.\n\nThis talk is about answering that for yourself, with two open-source Python tools: Inspect, the UK AI Security Institute's evaluation framework, and Petri, Anthropic's auditor built on top of it. Between them they let you judge an open-weight agent on your own traffic and on the failures you'd never think to collect, so you can put a number on the open-versus-frontier trade-off and decide on evidence.\n\nBy the end you'll be able to decide whether an open-weight model is good enough to replace a frontier API for your use case, and which one. You'll be able to: Build an evaluation from your own traffic, then let Petri's auditor extend it, so you test the cases you've seen and the ones you haven't. Score candidates against your current model on real agent work, across many turns and tool calls, not on a single tidy reply. Turn the result into a decision: a number on the quality-versus-cost, latency.","original_language":"eng","persons":["Emmanuel Okedele"],"view_count":10,"promoted":false,"date":"2026-10-23T16:30:00.000+02:00","release_date":"2026-11-11T00:00:00.000+01:00","updated_at":"2026-11-12T16:45:08.788+01:00","tags":["110878","2026","sps26","Data Science \u0026 More","Aula","sps26-eng"],"length":1690,"duration":1690,"thumb_url":"https://static.media.ccc.de/media/conferences/sps26/110878-fb4d780c-f6c0-4669-9fd3-e3e3788d4ca5.jpg","poster_url":"https://static.media.ccc.de/media/conferences/sps26/110878-fb4d780c-f6c0-4669-9fd3-e3e3788d4ca5_preview.jpg","timeline_url":"https://static.media.ccc.de/media/conferences/sps26/110878-fb4d780c-f6c0-4669-9fd3-e3e3788d4ca5.timeline.jpg","thumbnails_url":"https://static.media.ccc.de/media/conferences/sps26/110878-fb4d780c-f6c0-4669-9fd3-e3e3788d4ca5.thumbnails.vtt","frontend_link":"https://media.ccc.de/v/sps26-110878-from-open-weights-to-truste","url":"https://api.media.ccc.de/public/events/fb4d780c-f6c0-4669-9fd3-e3e3788d4ca5","conference_title":"Swiss Python Summit 2026","conference_url":"https://api.media.ccc.de/public/conferences/sps26","related":[],"recordings":[{"length":1690,"mime_type":"video/mp4","language":"eng","filename":"import-110878-eng-From_Open_Weights_to_Trusted_Models_hd.mp4","state":"new","folder":"h264-hd","high_quality":true,"width":1920,"height":1080,"updated_at":"2026-11-11T04:39:26.911+01:00","label":"eng 1080p","size":177,"recording_url":"https://cdn.media.ccc.de/events/sps26/h264-hd/import-110878-eng-From_Open_Weights_to_Trusted_Models_hd.mp4","url":"https://api.media.ccc.de/public/recordings/107350","event_url":"https://api.media.ccc.de/public/events/fb4d780c-f6c0-4669-9fd3-e3e3788d4ca5","conference_url":"https://api.media.ccc.de/public/conferences/sps26"},{"length":1690,"mime_type":"video/mp4","language":"eng","filename":"import-110878-eng-From_Open_Weights_to_Trusted_Models_sd.mp4","state":"new","folder":"h264-sd","high_quality":false,"width":720,"height":576,"updated_at":"2026-11-11T04:48:35.206+01:00","label":"eng 576p","size":60,"recording_url":"https://cdn.media.ccc.de/events/sps26/h264-sd/import-110878-eng-From_Open_Weights_to_Trusted_Models_sd.mp4","url":"https://api.media.ccc.de/public/recordings/107410","event_url":"https://api.media.ccc.de/public/events/fb4d780c-f6c0-4669-9fd3-e3e3788d4ca5","conference_url":"https://api.media.ccc.de/public/conferences/sps26"},{"length":1690,"mime_type":"audio/opus","language":"eng","filename":"import-110878-eng-From_Open_Weights_to_Trusted_Models_opus.opus","state":"new","folder":"opus","high_quality":false,"width":0,"height":0,"updated_at":"2026-11-11T04:48:38.575+01:00","label":"eng","size":17,"recording_url":"https://cdn.media.ccc.de/events/sps26/opus/import-110878-eng-From_Open_Weights_to_Trusted_Models_opus.opus","url":"https://api.media.ccc.de/public/recordings/107411","event_url":"https://api.media.ccc.de/public/events/fb4d780c-f6c0-4669-9fd3-e3e3788d4ca5","conference_url":"https://api.media.ccc.de/public/conferences/sps26"},{"length":1690,"mime_type":"audio/mpeg","language":"eng","filename":"import-110878-eng-From_Open_Weights_to_Trusted_Models_mp3.mp3","state":"new","folder":"mp3","high_quality":false,"width":0,"height":0,"updated_at":"2026-11-11T04:48:42.220+01:00","label":"eng","size":26,"recording_url":"https://cdn.media.ccc.de/events/sps26/mp3/import-110878-eng-From_Open_Weights_to_Trusted_Models_mp3.mp3","url":"https://api.media.ccc.de/public/recordings/107412","event_url":"https://api.media.ccc.de/public/events/fb4d780c-f6c0-4669-9fd3-e3e3788d4ca5","conference_url":"https://api.media.ccc.de/public/conferences/sps26"},{"length":1690,"mime_type":"video/webm","language":"eng","filename":"import-110878-eng-From_Open_Weights_to_Trusted_Models_webm-sd.webm","state":"new","folder":"webm-sd","high_quality":false,"width":720,"height":576,"updated_at":"2026-11-11T07:10:49.249+01:00","label":"eng 576p","size":75,"recording_url":"https://cdn.media.ccc.de/events/sps26/webm-sd/import-110878-eng-From_Open_Weights_to_Trusted_Models_webm-sd.webm","url":"https://api.media.ccc.de/public/recordings/107443","event_url":"https://api.media.ccc.de/public/events/fb4d780c-f6c0-4669-9fd3-e3e3788d4ca5","conference_url":"https://api.media.ccc.de/public/conferences/sps26"}]}