A comparison of four secure development assessment methodologies showed that there is no universal tool that provides an assessment, recommendations, and a ready-made action plan equally well. The practical differences are significant: some approaches help compare processes with those of other companies, others help build a development plan, while the GOST standard is better suited to checking compliance with requirements.
The author examined BSIMM, DSOMM, OWASP SAMM, and GOST R 56939-2024 using one hypothetical example. This was an audit simulation, not an assessment of a real organization: for each approach, the author completed an official or independently prepared questionnaire, determined a maturity level, and then checked whether it was possible to produce recommendations, a roadmap, and a heat map. The result depends not only on a company's practices, but also on whether the methodology offers a convenient way to turn answers into follow-up actions.
Four Different Ways to Look at Processes
BSIMM — Building Security In Maturity Model — describes secure development practices found in companies and shows how often they are used. In BSIMM15, the study covers 121 organizations. The model does not prescribe a mandatory implementation path: its strength is the ability to compare yourself with data from study participants. Practices are organized into four domains and three maturity levels. The report was previously published by Synopsys and is now published by Black Duck.
DSOMM, or DevSecOps Maturity Model, was developed by OWASP to assess the integration of security into DevOps. It includes five domains and five maturity levels, focuses on technical practices, and is intended to help identify gaps and ways to address them. An official online tool is available for assessments, although the author was unable to use it during the simulation described here and prepared a questionnaire in Excel instead. In the author's view, some practices may be too expensive or unsuitable for the capabilities of many Russian companies; the model also does not cover some stages of the development lifecycle.
OWASP SAMM covers the software lifecycle through five business functions, from governance to operations. Each practice is assessed on a scale from zero to three, and both the current and target levels can be recorded. Unlike the spreadsheets assembled independently for BSIMM and DSOMM, SAMM has an official Excel assessment tool and a guide. It can be used to create a roadmap. At the same time, the author found some practices vaguely worded, too costly, or poorly suited to the local context, while some SDLC stages were left out of scope.
GOST R 56939-2024 “Information Protection. Secure Software Development. General Requirements” is not formally a maturity assessment methodology. It is a standard containing 25 processes and requirements for developers at different stages, from design to software decommissioning. Artifacts are important for verification: compliance with requirements must be supported by documents, reports, tracker entries, or version control system records. The standard expanded the requirements of the previous 2016 edition and is adapted to the Russian market, but it contains no official mechanism for calculating a maturity level.
Assessment Using a Hypothetical Company
For comparison, the author modeled ООО «Лиса и лосЬ» — a company that develops mostly CRM systems to order for internal and external clients. Its pipeline runs on GitLab, artifacts are stored in Nexus, and code is checked using SonarQube and OSV-scanner. A secure development policy exists, but it is based on GOST R 56939-2016. In the simulated profile, there is no training in secure development, pentesting, or WAF.
The author used five criteria, with a total maximum of 47 points. The first four concerned how easy it was to determine maturity, obtain recommendations, build a roadmap, and create a heat map. The first criteria were worth up to ten points; ease of use was assessed subjectively, while the availability of tools for recommendations and maps was scored on a scale from no such capability to a ready-made open-source or proprietary solution. The final criterion assessed coverage of the seven SDLC stages: requirements gathering and analysis, planning and risk analysis, design, development, testing, deployment, and support.
The author could not find a BSIMM tool, and the official DSOMM web application did not work in his attempt, so he created the questionnaire and calculations for both models himself in Excel. For GOST, the assessment used an internal tool from the company “Bastion,” which is not publicly available. This limitation matters when interpreting the scores: the GOST assessment cannot be reproduced using an open official calculator. Sometimes the technical part of an assessment starts not with analyzing processes, but with creating the spreadsheet that was supposed to help conduct that analysis.
What the Results Showed
The hypothetical company's maturity was assessed at 18% under BSIMM, 2 out of 5 under DSOMM, 0,55 out of 3 under SAMM, and 29% under GOST R 56939-2024. These values cannot be directly compared: the methodologies use different scales, and the assessment itself depends on how each one is structured. This is more a profile under each individual approach than four results from a single standardized test.
By total score, GOST led the comparison with 35 points out of 47, while SAMM scored 34. A one-point difference does not make one methodology unequivocally better: their goals differ. GOST defines requirements and criteria for validating them with artifacts in detail, covers the SDLC stages, and provides a basis for an in-depth process review. But assessors will have to handle the calculations and roadmap themselves if they need a ready-made tool in that format.
SAMM has a different advantage: its official questionnaire helps assess maturity and create a flexible roadmap in one tool. The author therefore considers it suitable when a company is planning an AppSec strategy and wants to see a direction for development. The trade-off is that some practices may be costly or unsuited to the conditions of Russian companies, and some lifecycle stages are not fully covered.
DSOMM is worth considering when the main focus is DevSecOps and technical practices. Explanations accompanying the questions help make them easier to interpret consistently, and the model is supported by the community. However, its official online mechanism could not be used in this assessment. The author notes that specialist forums contain reports of people being able to use it, so his experience should not be taken as proof that the tool is generally broken. The limited SDLC coverage and the cost or unsuitability of some practices should also be taken into account.
BSIMM is best suited to comparing a company's own practices with data from other companies: among the methodologies considered, it is the only one with open data for such comparisons, including annual reports and a selection of popular practices. The model is less convenient for internal self-assessment because there is no official calculation algorithm. In addition, practices have to be adapted to local conditions, and in the author's view some may be unsuitable for many Russian organizations.
The practical choice should therefore be tied to the goal: GOST for preparing for a requirements-based audit, SAMM for an AppSec strategy and roadmap, DSOMM for the technical side of DevSecOps, and BSIMM for comparison with companies in the study. No single approach covers every task at once. For a comprehensive audit, the author recommends combining several methodologies and adapting the criteria to the specific organization, bearing in mind that a maturity assessment is only the beginning of the work, not a ready-made implementation plan.
