{"guid":"69dc1dd7-b5c4-4f86-921f-3bf98d0b7569","title":"What LLMs Can Do in Pentesting and Code Security","subtitle":null,"slug":"god2026-110754-what-llms-can-do-in-pente","link":"https://c3voc.de","description":"Large language models are starting to change both sides of application security: offensive work such as black-box penetration testing, and defensive work such as code review and vulnerability detection in source code.\n\nIn this talk, I present practical lessons from my academic and independent work in both areas: AutoPentest, my research on autonomous black-box pentesting with LLM agents; Vuldra, my work on LLM-assisted static code analysis; and my recent hands-on evaluation of OpenAI Codex Security on a real open-source project.\n\nOn the offensive side, I show what happens when an LLM agent is asked to carry out a black-box penetration test with a high degree of autonomy. I explain the architecture behind AutoPentest, including specialized worker agents structured around OWASP Top 10-relevant web vulnerability areas, and show where the system still struggles in longer attack chains.\n\nIn my evaluation on three Hack The Box machines, AutoPentest completed 15 to 26 percent of subtasks and slightly outperformed a manual ChatGPT-based baseline on one target.\n\nOn the defensive side, I explored two ways of using LLMs for code security testing.\n\nFirst, I discuss Vuldra as an example of how LLMs can be used for static code analysis, integrating them with traditional SAST tools.\n\nSecond, I present my evaluation of Codex Security on TorMap, where I looked at real findings, proposed fixes, and practical limits in a developer workflow.\n\nThe main message of the talk is simple: LLMs can already help security teams on both the attacker and defender side, but their strengths and weaknesses are different.\n\nAttendees will leave with a realistic view of where LLMs are already useful, where they still fail, and how to evaluate such tools in their own environment without treating them as magic.\n\nLicensed to the public under https://creativecommons.org/licenses/by-sa/4.0/","original_language":"eng","persons":["Julius Henke"],"view_count":10,"promoted":false,"date":"2026-09-24T12:05:00.000+02:00","release_date":"2026-09-24T00:00:00.000+02:00","updated_at":"2026-09-24T17:00:04.658+02:00","tags":["110754","2026","god2026","Saal Baden","god2026-eng","god2026","Day 1"],"length":2890,"duration":2890,"thumb_url":"https://static.media.ccc.de/media/events/god/2026/110754-69dc1dd7-b5c4-4f86-921f-3bf98d0b7569.jpg","poster_url":"https://static.media.ccc.de/media/events/god/2026/110754-69dc1dd7-b5c4-4f86-921f-3bf98d0b7569_preview.jpg","timeline_url":"https://static.media.ccc.de/media/events/god/2026/110754-69dc1dd7-b5c4-4f86-921f-3bf98d0b7569.timeline.jpg","thumbnails_url":"https://static.media.ccc.de/media/events/god/2026/110754-69dc1dd7-b5c4-4f86-921f-3bf98d0b7569.thumbnails.vtt","frontend_link":"https://media.ccc.de/v/god2026-110754-what-llms-can-do-in-pente","url":"https://api.media.ccc.de/public/events/69dc1dd7-b5c4-4f86-921f-3bf98d0b7569","conference_title":"German OWASP Day 2026","conference_url":"https://api.media.ccc.de/public/conferences/god2026","related":[],"recordings":[{"length":2890,"mime_type":"video/mp4","language":"eng","filename":"god2026-110754-eng-What_LLMs_Can_Do_in_Pentesting_and_Code_Security_hd.mp4","state":"new","folder":"h264-hd","high_quality":true,"width":1920,"height":1080,"updated_at":"2026-09-24T14:04:48.577+02:00","label":"eng 1080p","size":239,"recording_url":"https://cdn.media.ccc.de/events/god/2026/h264-hd/god2026-110754-eng-What_LLMs_Can_Do_in_Pentesting_and_Code_Security_hd.mp4","url":"https://api.media.ccc.de/public/recordings/105369","event_url":"https://api.media.ccc.de/public/events/69dc1dd7-b5c4-4f86-921f-3bf98d0b7569","conference_url":"https://api.media.ccc.de/public/conferences/god2026"},{"length":2890,"mime_type":"audio/opus","language":"eng","filename":"god2026-110754-eng-What_LLMs_Can_Do_in_Pentesting_and_Code_Security_opus.opus","state":"new","folder":"opus","high_quality":false,"width":0,"height":0,"updated_at":"2026-09-24T14:07:02.882+02:00","label":"eng","size":26,"recording_url":"https://cdn.media.ccc.de/events/god/2026/opus/god2026-110754-eng-What_LLMs_Can_Do_in_Pentesting_and_Code_Security_opus.opus","url":"https://api.media.ccc.de/public/recordings/105370","event_url":"https://api.media.ccc.de/public/events/69dc1dd7-b5c4-4f86-921f-3bf98d0b7569","conference_url":"https://api.media.ccc.de/public/conferences/god2026"},{"length":2890,"mime_type":"audio/mpeg","language":"eng","filename":"god2026-110754-eng-What_LLMs_Can_Do_in_Pentesting_and_Code_Security_mp3.mp3","state":"new","folder":"mp3","high_quality":false,"width":0,"height":0,"updated_at":"2026-09-24T14:07:07.052+02:00","label":"eng","size":44,"recording_url":"https://cdn.media.ccc.de/events/god/2026/mp3/god2026-110754-eng-What_LLMs_Can_Do_in_Pentesting_and_Code_Security_mp3.mp3","url":"https://api.media.ccc.de/public/recordings/105371","event_url":"https://api.media.ccc.de/public/events/69dc1dd7-b5c4-4f86-921f-3bf98d0b7569","conference_url":"https://api.media.ccc.de/public/conferences/god2026"},{"length":2890,"mime_type":"video/webm;codecs=av01","language":"eng","filename":"god2026-110754-eng-What_LLMs_Can_Do_in_Pentesting_and_Code_Security_av1-sd.webm","state":"new","folder":"av1-sd","high_quality":false,"width":720,"height":576,"updated_at":"2026-09-24T14:11:52.133+02:00","label":"eng 576p","size":72,"recording_url":"https://cdn.media.ccc.de/events/god/2026/av1-sd/god2026-110754-eng-What_LLMs_Can_Do_in_Pentesting_and_Code_Security_av1-sd.webm","url":"https://api.media.ccc.de/public/recordings/105373","event_url":"https://api.media.ccc.de/public/events/69dc1dd7-b5c4-4f86-921f-3bf98d0b7569","conference_url":"https://api.media.ccc.de/public/conferences/god2026"},{"length":2890,"mime_type":"video/mp4","language":"eng","filename":"god2026-110754-eng-What_LLMs_Can_Do_in_Pentesting_and_Code_Security_sd.mp4","state":"new","folder":"h264-sd","high_quality":false,"width":720,"height":576,"updated_at":"2026-09-24T14:08:31.167+02:00","label":"eng 576p","size":92,"recording_url":"https://cdn.media.ccc.de/events/god/2026/h264-sd/god2026-110754-eng-What_LLMs_Can_Do_in_Pentesting_and_Code_Security_sd.mp4","url":"https://api.media.ccc.de/public/recordings/105372","event_url":"https://api.media.ccc.de/public/events/69dc1dd7-b5c4-4f86-921f-3bf98d0b7569","conference_url":"https://api.media.ccc.de/public/conferences/god2026"},{"length":2890,"mime_type":"video/webm;codecs=av01","language":"eng","filename":"god2026-110754-eng-What_LLMs_Can_Do_in_Pentesting_and_Code_Security_av1-hd.webm","state":"new","folder":"av1-hd","high_quality":true,"width":1920,"height":1080,"updated_at":"2026-09-24T14:23:06.353+02:00","label":"eng 1080p","size":201,"recording_url":"https://cdn.media.ccc.de/events/god/2026/av1-hd/god2026-110754-eng-What_LLMs_Can_Do_in_Pentesting_and_Code_Security_av1-hd.webm","url":"https://api.media.ccc.de/public/recordings/105374","event_url":"https://api.media.ccc.de/public/events/69dc1dd7-b5c4-4f86-921f-3bf98d0b7569","conference_url":"https://api.media.ccc.de/public/conferences/god2026"},{"length":2890,"mime_type":"video/webm","language":"eng","filename":"god2026-110754-eng-What_LLMs_Can_Do_in_Pentesting_and_Code_Security_webm-sd.webm","state":"new","folder":"webm-sd","high_quality":false,"width":720,"height":576,"updated_at":"2026-09-24T14:33:32.973+02:00","label":"eng 576p","size":95,"recording_url":"https://cdn.media.ccc.de/events/god/2026/webm-sd/god2026-110754-eng-What_LLMs_Can_Do_in_Pentesting_and_Code_Security_webm-sd.webm","url":"https://api.media.ccc.de/public/recordings/105378","event_url":"https://api.media.ccc.de/public/events/69dc1dd7-b5c4-4f86-921f-3bf98d0b7569","conference_url":"https://api.media.ccc.de/public/conferences/god2026"},{"length":2890,"mime_type":"video/webm","language":"eng","filename":"god2026-110754-eng-What_LLMs_Can_Do_in_Pentesting_and_Code_Security_webm-hd.webm","state":"new","folder":"webm-hd","high_quality":true,"width":1920,"height":1080,"updated_at":"2026-09-24T14:35:07.175+02:00","label":"eng 1080p","size":241,"recording_url":"https://cdn.media.ccc.de/events/god/2026/webm-hd/god2026-110754-eng-What_LLMs_Can_Do_in_Pentesting_and_Code_Security_webm-hd.webm","url":"https://api.media.ccc.de/public/recordings/105379","event_url":"https://api.media.ccc.de/public/events/69dc1dd7-b5c4-4f86-921f-3bf98d0b7569","conference_url":"https://api.media.ccc.de/public/conferences/god2026"}]}