展示HN:我创建了一个由AI驱动的Python测试套件,它可以自动编写测试。
我一直在进行一个项目,迫不及待想与Hacker News社区分享。这是一个基于AI的Python测试套件,采用混合AI方法,能够自动生成全面的单元测试、执行模糊测试,甚至进行变异测试,以评估您现有测试套件的质量。
*问题*
作为一名开发者,我一直发现编写和维护一个稳健的测试套件是软件开发中最耗时和最具挑战性的方面之一。通常很难想到所有可能的边界情况,并确保您的测试能够有效捕捉到漏洞。
*解决方案*
为了解决这个问题,我创建了一个MCP服务器,利用谷歌的Gemini AI和BAML(边界机器学习)提供一套智能测试工具。该服务器基于FastMCP框架构建,可以轻松集成到您现有的工作流程中。
*技术深度分析*
以下是关键功能的详细介绍及其工作原理:
* *混合AI方法:* 该项目采用混合AI方法,结合了BAML和Gemini的优势。BAML用于结构化测试生成,确保输出始终以一致且可解析的格式呈现。Gemini则利用其强大的语言理解能力,生成富有创意和挑战性的测试用例。
* *智能单元测试生成:* 单元测试生成器使用AI为您的Python代码创建全面的测试套件。它自动识别边界情况、错误条件和其他潜在的漏洞来源。生成的测试使用`unittest`框架编写,并包含适当的断言和错误处理。
* *AI驱动的模糊测试:* 模糊测试器利用AI生成多样化的输入,以测试您函数的稳健性。它可以生成从简单的边界情况到格式错误的数据和大输入,帮助您识别潜在的崩溃和其他意外行为。
* *高级覆盖率测试:* 覆盖率测试器结合AST分析和AI驱动的测试生成,达到最大代码覆盖率。它识别代码中的所有可能分支、循环和异常路径,然后生成测试以覆盖每一个。
* *智能变异测试:* 变异测试器使用自定义的基于AST的变异引擎来评估您现有测试套件的质量。它生成一系列小的语法变化(变异)并检查您的测试是否能够检测到这些变化。这有助于您识别测试覆盖的空白,并提高测试的整体有效性。
*行动呼吁*
我仍在积极开发该项目,非常希望能听到您的反馈。您可以在GitHub上找到源代码: [https://github.com/jazzberry-ai/python-testing-mcp](https://github.com/jazzberry-ai/python-testing-mcp)
我特别希望听到您对以下问题的看法:
* 您希望在套件中添加其他哪些测试工具?
* 您在使用该工具时发现过有趣的漏洞或边界情况吗?
* 您对改进提示或AI模型有任何建议吗?
感谢您的阅读,期待您的反馈!
查看原文
I've been working on a project that I'm excited to share with the Hacker News community. It's an AI-powered Python testing suite that uses a hybrid AI approach to automatically generate comprehensive unit tests, perform fuzz testing, and even conduct mutation testing to assess the quality of your existing test suites.<p>*The Problem*<p>As a developer, I've always found writing and maintaining a robust test suite to be one of the most time-consuming and challenging aspects of software development. It's often difficult to think of all the possible edge cases and to ensure that your tests are actually effective at catching bugs.<p>*The Solution*<p>To address this, I've created an MCP server that leverages both Google's Gemini AI and BAML (Boundary ML) to provide a suite of intelligent testing tools. The server is built on the FastMCP framework and can be easily integrated into your existing workflow.<p>*Technical Deep Dive*<p>Here's a breakdown of the key features and how they work:<p>* *Hybrid AI Approach:* The project uses a hybrid AI approach that combines the strengths of both BAML and Gemini. BAML is used for structured test generation, ensuring that the output is always in a consistent and parseable format. Gemini is used for its powerful language understanding capabilities, which allows it to generate creative and challenging test cases.<p>* *Intelligent Unit Test Generation:* The unit test generator uses AI to create a comprehensive suite of tests for your Python code. It automatically identifies edge cases, error conditions, and other potential sources of bugs. The generated tests are written using the `unittest` framework and include proper assertions and error handling.<p>* *AI-Powered Fuzz Testing:* The fuzz tester uses AI to generate a diverse range of inputs to test the robustness of your functions. It can generate everything from simple edge cases to malformed data and large inputs, helping you to identify potential crashes and other unexpected behavior.<p>* *Advanced Coverage Testing:* The coverage tester uses a combination of AST analysis and AI-powered test generation to achieve maximum code coverage. It identifies all possible branches, loops, and exception paths in your code and then generates tests to cover each of them.<p>* *Intelligent Mutation Testing:* The mutation tester uses a custom AST-based mutation engine to assess the quality of your existing test suite. It generates a series of small, syntactic changes to your code (mutations) and then checks to see if your tests are able to detect them. This helps you to identify gaps in your test coverage and to improve the overall effectiveness of your tests.<p>*Call to Action*<p>I'm still actively developing the project, and I would love to get your feedback. You can find the source code on GitHub: <a href="https://github.com/jazzberry-ai/python-testing-mcp" rel="nofollow">https://github.com/jazzberry-ai/python-testing-mcp</a><p>I'm particularly interested in hearing your thoughts on the following:<p>* Are there any other testing tools that you would like to see added to the suite?
* Have you found any interesting bugs or edge cases using the tool?
* Do you have any suggestions for improving the prompts or the AI models?<p>Thanks for reading, and I look forward to hearing from you!