The software development area is one of those that most benefit from the use of open-source tools. After all, it was there that the first open-source solutions emerged. Even though the variety of options has increased exponentially, development continues to take advantage of this tool the most.

And no less. For each step of the process (analysis, design, implementation, testing, and maintenance), there is a huge range of tools for optimizing time and resources. Even IT management takes advantage of this.

Other open-source solutions also help with modeling, agile development tools, diagram creation, and numerous related activities. It is worth mentioning that these tools can be applied to optimize the creation of different products and services, not just software.

In this article, you will see the advantages of using open source tools that provide greater quality in the software development process.

But first, let’s understand a little bit about open source.

Open source is any computer program that can be used, copied, studied, modified, and redistributed without any restrictions.

The term open source is used to refer to all software whose source code is open to anyone. This means that third parties can analyze, audit and in some cases even make changes to meet your needs or make improvements. Open source should not be confused with Free Software, which has similar characteristics: basically all free software must be open source, but the opposite is not always true.

But open source doesn’t just mean access to the source code. Below we list some criteria to define an open-source solution or tool:

• Open source code;
• Possibility to create derivative works;
• The integrity of the author’s source code;
• No discrimination against people or groups;
• No discrimination against fields of action;
• License distribution;
• The license must not be specific to a product;
• The license must not restrict other software;
• The license must be technology-neutral.

In the beginning, the main open-source software created was focused on software development, C compilers like GCC, text editors like Emacs, among others. Subsequently, complete operating systems and other tools emerged, but the development area continued to be one of the areas with the greatest free resources in the software market.

In the topics below, we will see options that can help developers to develop software with the support of open source tools, which will enable the delivery of software with more quality and agility.

Open-source Libraries, routines, and system components to help developers

A common way to use open source in development is the inclusion of standard routines, already developed and tested in the market, which because they are ‘open source’ can be incorporated into your software code to avoid the need to develop it internally.

These components are used, for example, to transfer information between databases, do some image processing, perform a validation routine, etc.

Open source Database systems

The resources of open source databases have reached a high level of use thanks to the growing number of companies that use them for large-scale projects.

MySQL is one of the most widely used open-source relational databases. Another example is PostgreSQL, an object-relational database that supports macOS Server, most Linux, and also Windows distributions.

Quality control automation tools

Talking about automation options in the quality control process, there are a few options on the market. One of them is Selenium, formed by a set of open-source software testing automation tools. It supports mobile testing, several programming languages, different operating systems, and browsers. Watir (an acronym for Web Application Testing in Ruby) is an open-source Ruby library for performing simple automated tests. It works with browsers like Edge, Firefox, Chrome, and Safari. These are valid resources, but there are other, far more complete and efficient solutions on the market, provided by companies specializing in software quality – for example, which scan all open software components used in coding an application, and which then list all the vulnerabilities,

With all this, using tools that help to monitor the open software components that are used by the company, ensuring that they are in their appropriate versions and covering security issues properly, is essential.

After all, it is practically impossible to work with software development without having the support of at least one open-source tool.

Open source Growth and consolidation

After the appearance of the first projects, throughout the 2000s the open-source community continued to grow, and the idea that open software ended up being more robust and performative was consolidated. Because of this, most software development technologies that have emerged in recent years have become or have been conceived as open: Docker, NodeJS, MongoDB, ReactJS, Kubernetes, VueJS, NPM, TensorFlow, .NET Core, and several others. In addition, there were also very rich open-source library packages for free use, such as NPM for Javascript, Maven for Java, and pip for Python, each with thousands of open-source libraries that are used in thousands of other projects.

Over the years, several companies have found business models that fit the open-source paradigm. The most famous example is perhaps that of Red Hat, which decided to offer support services at the enterprise level for the use of GNU / Linux distributions and worked very well: After years of work and adjustments to its business model, in 2019 Red Hat was acquired by IBM. In addition to Red Hat, other companies operating in the open source ecosystem are on the same promising path, such as Elastic, MuleSoft, and MongoDB.

Following the principle that open technologies ended up becoming of higher quality due to the organic performance of the open-source community, several companies decided to open the code of their tools in order to develop them better instead of selling use licenses and limiting the growth of products. The most notable examples would be Facebook with ReactJS, Google with TensorFlow, and Microsoft with .NET Core.

The importance of open source to the development market

Currently, in the context of tools for software development, we can already say that open source has become the standard. Especially for developers, who daily use software libraries that have been developed openly by other programmers (NPM packages, Maven, etc).

It is very important to understand how these solutions were developed, how to make changes to them, and how to adapt them to your specific needs. Without a doubt, any developer who has a history of significant contributions to open source projects will have a much more attractive curriculum, as the ability to contribute demonstrates consolidated technical knowledge and commitment to the growth of the software development community and ecosystem.

As the last few years have shown, the joint creation of software brings excellent results: more reliable solutions, of the highest quality and open for use. Even the giant tech companies have realized the benefits of the model, which despite being counterintuitive at first, is consolidating itself as the standard in the community.

It was realized that there is no point in developing a solution and hiding it behind a patent or something similar because over time it is inevitable that the community develops a better alternative in a collaborative way. Therefore, it is much more intelligent to “surf the wave” and gain a leading role in the open-source scene, so that both the community and the creators of the software can benefit.

Security advantages in open source software

It sounds counterintuitive, but it’s true: open source software is more secure. When many people work on the same source code, there are many more eyes checking and correcting security holes. When one is discovered, it doesn’t take long for one of those thousands or millions of developers to find the solution.

Developers rely heavily on open-source software and companies are comfortable with the main open-source projects that have large groups maintaining them. This is the great advantage of using open-source software because in addition to being less expensive, there are more people working to maintain and evolve it.

Another security advantage of open source is that, if there is a problem, a company can open and fix it immediately. If the code is licensed under proprietary agreements, they generally need to wait for suppliers to respond.

And about open source support

The community behind open source software is often participatory and collaborative – there are answers to any question.

But if this open documentation is still not enough, there are companies that specialize in supporting open source. That is, you can use the money saved with the license to guarantee the availability of your system.

Adoption of open-source tools

While open-source tools are becoming more and more popular, still there is a large portion of software developers who are not quick to adopt such tools for various reasons. Some of those reasons are due to lack of awareness (they do not know they exist), or they don’t know how to install, integrate, maintain or use them due to lack of technical knowledge and lack of time.

For example, you may find a very knowledgeable C/C++ developer who has excellent experience and knowledge with software development with C/C++ on windows. However, a huge portion of open source tools out there are developed on Linux and are offered to those who develop on Linux. The windows software developer might come across a very attractive open-source tool he might want to use that was developed on Linux but is now facing the need to learn how to use the tool on Windows. More than often such developers won’t bother to use such a tool. This is an example of the exclusion of an entire segment of software developers.

Although not said or announced anywhere, the writing on the wall is basically “learn Linux, spend hours learning each new open-source tool and get your hands dirty”. That’s fair, but if you think about it, most tools are installed, integrated, and rarely modified afterward. It seems like a HUGE waste of time and efficiency to repeat this learning curve again and again from one developer to another. That is why SoftaCheck was created. This is an online tool that makes open-source tools accessible to all developers while preventing time wastage and letting developers focus on what they like and do best – developing. SoftaCheck, for example, lets software developers run static analysis on their code with clang-tidy and cppcheck – both open-source static analysis tools. SoftaCheck also offers doxygen which lets you generate documentation for your code automatically. SoftaCheck does all of this in the cloud through a GitHub integration which can be installed with only a few mouse clicks.

The post Open-Source Software Development Tools In 2021 appeared first on SoftaCheck.

Code Documentation using Doxygen

“Comments often are used as a deodorant.”

— Martin Fowler and Kent Beck, Refactoring, page 87

Forward

The agile software manifesto states:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That it values items to the left more (emphasis mine) than items to the right. Of the four universal statements, I believe the second one is the most misunderstood. Without working software, comprehensive documentation is null and void. The operative word here is comprehensive. It takes time and effort to create comprehensive documentation. But it should never be an excuse to not create code documentation.

What use is code documentation?

I believe documenting the code is different from just commenting on it. If you are trying to reduce the “odour” of code by commenting on it you better refactor the code to make it more understandable. I believe code documentation should:

Provide information that cannot be expressed by the code

So, what kind of information does code documentation provide that cannot be expressed by code?

• Requirement traceability: A unit of code – be it a function, class, module, or component – should be traceable to one or more requirements. Requirement traceability is required to answer two fundamental questions: are we building the right product? Are we building the product right? Code documentation can embed requirement identifies that enable forward and backward tracing to code.
• Design by contract: This method of software development has its roots in the formal verification of software. A contract is a unit of code. We have the following three questions that need to be answered by this contract:
  • What does the contract expect?
  • What does the contract guarantee?
  • What does the contract maintain?

Such strict rules ensure software correctness and can be written by code comments – especially in languages that have no native support for contracts.

• 4+1 architectural view model: In the 4+1 architectural view model, the development view is concerned with code. This is the implementation view. There is no reason why code documentation cannot allude to other views – especially the dynamic view and the deployment view.
• As a heads-up to the maintainers: I cannot describe it any better than the image I found on the internet:
  

  Heads Up To Code Maintainers (an indication that code must be refactored)

• Regulatory requirement: In some domains like medical image processing, code documentation is mandatory as it is a requirement to pass Food and Drugs Authority (FDA) regulations. In such situations, the metric of code to comment ratio will be enforced.
• API documentation: Systems which expose services through APIs (REST-based) need to provide API documentation. This documentation is exposed to the clients as API documentation. It helps clients to understand the functionality of exposed APIs and consume them as per specification. Also, as APIs change over time (with new functionalities getting added, older ones getting depreciated) API documentation should state the versioning information.

Doxygen

In the previous section, I believe I have driven home the point that code documentation is important. Although code and code documentation both goes into the source code file, it would be unwise to read the file for documentation. Separation of concerns is what we need:

Everything Under Version Control (Documentation and Code)

Doxygen is a well-known tool to generate documentation from annotated sources for different programming languages like:

• C and C++
• Objective C
• C#
• PHP
• Java
• Python
• Fortran
• ..etc

Doxygen can generate both Online (HTML hosted on a server viewable through a browser, e.g. D-Bus documentation https://dbus.freedesktop.org/doc/api/html/index.html) and offline documentation (in RTF, PDF, PostScript, or LaTeX formats).

Doxygen is a mature open-source project and has been around for more than 20 years. It is well maintained because it solves a vital problem of generating code documentation from annotated files thereby delivering value.

Doxygen Example

For this demonstration, I am creating a blogging system in C++ to show off some code. Here is one header:

BlogEntry.h:

BlogEntry.h – Doxygen example (how to write comments)

BlogEntry.cpp:

BlogEntry.cpp – Doxygen example (how to write comments)

As you can see, I have documented my code using Doxygen markups. To create an html documentation that I can host online two paths could be taken.

• Path 1: Download and Install doxygen myself, configure it myself, run it myself and upload the documentation to the cloud to a hosting service myself.
• Path 2: Use SoftaCheck that takes care of all of that and does it automatically in the cloud with a selected GitHub repository and stores the documentation in the cloud with a password-protected link.

Here is the experience with Path 1:

1. Download and install Doxygen. Once installation is done, you should be able to print the version of Doxygen from the command line using doxygen -v
   

   Doxygen Installation Step 1

2. Create a configuration file in the same directory of your sources by the command doxygen -g <configfilename>
   

   Doxygen Installation Step 2

3. The created democonfig file needs some edits. Here is what you need to do:
   

   Doxygen Installation Step 3

4. Now run the command: doxygen democonfig
   

   Doxygen Installation Step 4

You will see Doxygen preprocessing and parsing the headers and sources of your project. Once done, you can check the generated documentation using a web browser navigating to .\html\index.html page.

Let us give a project number and see what changes.

PROJECT_NUMBER         = 3.14

And regenerate the documentation.

5. Here is how the documentation for the class would look like:
   

Here is how the documentation for the header would look like:

Adding versioning to the documentation is very important – as important as versioning the source code itself. Keeping documentation and code versions in sync manually is a pain. To alleviate the pain, we in Softacheck have an excellent solution.

Softacheck solution for code documentation

Softacheck is a GitHub app available through the GitHub marketplace which allows you to automate your static code analysis and code documentation process within GitHub workflow. There is no local installation required on developer machines. This ensures there are no circumvention possibilities on Softacheck as it is run on every commit.

Once you configure Softacheck to run on your repositories, the Doxygen code documentation will be generated and stored in servers managed by Softacheck. We ensure that only members of the repository shall have access to the documentation. GitHub credentials have to be supplied for a successful login. Since code documentation generation does not involve any manual steps, up-to-date documentation is always ensured. Also, on the pull request page, a link to the documentation page will be available. Most importantly, all past revisions of the documentation are maintained so that you can always view your earlier code versions with synced documentation. Isn’t that a great USP?

Also, if you do want to use your own doxygen configuration file to have more control over your doxygen output you can simply include the configuration file in the repository and SoftaCheck will use it as the default configuration file for your doxygen settings.

And yes, for a limited time, we are giving our excellent app for you to try – for free!

Afterword

Comments definitely should not be used as deodorants. Code should never smell. If there are design or code smells in the project, use that as a guiding principle to refactor your code. Maintain code documentation. It has a direct impact on the internal quality of your system. You must maintain your code as well as your documentation making sure that they are in sync. I will be pleased if you try out Softacheck for this purpose!

The post Doxygen – What is Code Documentation and How to Generate It? appeared first on SoftaCheck.

Static Analysis Four Stages of Competence

The “four stages of competence” is a well-known learning model that depicts stages that a learner passes through while acquiring a skill. If you have never heard of static code analysis, this blog post is for you. By the end of this blog post, if you decide to learn more about static code analysis, you have successfully transitioned from the “I don’t know that I don’t know” state to the “I know that I don’t know” state.

At their heart, compilers are tools that transform human-readable text to machine-readable code. If the compiler doesn’t encounter any error during this transformation from source code an executable is born. 99% of the time, a program is born with logical errors/vulnerabilities/functional defects that the compiler knows nothing about. (What about the remaining 1%? They are miracle births that are celebrated far and wide.)

Not only are compilers faithful in taking programmer’s instructions but also, they help to give hints to improve a program. These hints depend on compiler settings and are known as warning levels. These warnings are a result of static code analysis done by the compiler during program compilation.

The “90/10” rule is a well-known paradigm that applies to many areas of computer science. For example, 90 percent of a program execution time is spent in 10 percent of the code. Extrapolating this rule to warnings thrown by the compiler we can claim that 90 percent of programmers fix only 10 percent of compiler warnings. They are just warnings, I hear you say!

Bad things happen when well-intended warnings given by the compiler are not heeded to. Programmers make subtle mistakes while writing a program that cannot be caught by the compiler. These mistakes manifest themselves as software errors? or vulnerabilities. ?Let us go through some such examples:

Ariane 5 Disaster

What can be worse than losing a rocket 30 seconds into its launch blowing up 370 million dollars in tax payer’s money? All because the compiler tried to cram a 64-bit value into a 16-bit address.

A 64-bit value can be a very large number (a number up to 2^64 -1 to be precise). Clearly, this cannot fit into a 16-bit address space but the compiler generated instructions to do just that! You can read more about the disaster here.

Therac-25 radiation overdose

Therac-25 is the story of how a machine intended to save the lives of patients ultimately became their killer – all because of bugs in its software. Race conditions and integer overflows resulted in the machine malfunctioning thereby sending high doses of radiation.

Race conditions happen when multiple threads executing a block of code are not properly synchronized. For example:

Data races gave an incorrect result because of concurrent execution of threads that are not synchronized. Such bugs are non-deterministic and quite hard to track.

Integer overflows happen when an operation results in values that are outside of the range that can be represented with a given number of digits. Assume that a 16-bit unsigned integer is holding its maximum value 65535 like so:

If we try to add 1 to this number an integer overflow happens because 65536 cannot be represented in a 16-bit integer. This overflow causes undefined behavior with signed integers and can cause data to leak to other memory space (with an unsigned integer, however, overflow behavior is defined and predictable and will not cause a problem).

Both race conditions and integer overflows are not normally detected during compilation – although a special class of Integer overflows involving only constants can be detected.

Heartbleed – vulnerability in OpenSSL

This is one of those rare bugs which has a webpage of its own! OpenSSL is a cryptographic library used to secure information. It is opensource and well supported in all modern operating systems. A missing bounds-check in the handling of the TLS heartbeat extension can be exploited to reveal sensitive information. Bound-check omission, though not caught by the compiler can be caught by a static code analysis tool. A critical software vulnerability can even be weaponized. The most famous of such cases is that of Stuxnet where industrial automation software was specifically targeted to deliver a worm that disrupted the programable logic controllers thereby acting as a weapon.

Aside: What is bound check and why is it important?

Arrays are variables that store values of the same type in contiguous memory locations. Assume we have an array like so:

The bounds of the array are (0, 7) both inclusive. Some languages like C# or Java check if the array access is within this bound. Languages like C and C++ let the programmer deal with array access and there is no bound check added by the compiler. This leads to subtle bugs that can be caught during static code analysis.

There are different ways in which the software industry tries to deal with such bugs. One is the tacit understanding that not all bugs can be eliminated. The following can and should be done to minimize software errors:

• Code reviews – The practice of overseeing code before it gets into production is a great way of arresting bugs. The effectiveness of this approach depends greatly on the capability of the reviewer.
• Software testing – More than 50% of the time in software projects is spent on testing. Testing can catch bugs and automated test runs to ensure the bugs don’t regress. But testing is costly and also can lead to a false sense of security.
• Static code analysis – Unlike code reviews that need manual reviewers, static code analysis uses tools to check programs. This checking can even be integrated into the nightly builds to generate daily build reports. A disadvantage would be the cost associated with the tool.

What is static code analysis?

During compilation source code is transformed into intermediate representations like Abstract Syntax Tree (AST) and Control Flow Graph (CFG). Compilers use these intermediate representations to run data flow analysis (DFA) algorithms to do code optimizations. During the code optimization stage, it is possible to determine the unused variables and unused code (dead code). The primary goal of a compiler is to transform such intermediate representations to executable code, whereas the primary goal of a static code analysis tool is to use the intermediate representations to find issues in the code.

What can static code analysis do for you?

Static code analysis can

• Detect code that deviates from a coding standard (e.g. MISRA C)
• Detect code that can lead to resource or memory leaks
• Detect code that can lead to null pointer dereferencing
• Detect concurrency issues in code leading to race conditions
• Detect incorrect use of APIs
• Detect conditionals that always evaluate to either true or false
• Detect operator precedence issues
• More…

(Check out the appendix section for more details on some of these issues.)

Most static code analysis tools are well integrated into the development environment. This gives the programmer a chance to run static code analysis on demand. Mostly this opportunity comes after the “last elusive bug” is found or the last customer feature is done – which is never. ?

Hence the ideal way to run static code analysis is to integrate it with the source control management and its nightly build setup. Our tool Softacheck lets you seamlessly analyze C and C++ code hosted on GitHub. Unlike some static code analysis tools that are prohibitively expensive Softacheck is currently free! ?

Appendix

1. AST: Abstract Syntax Tree is data structure obtained as a result of lexing and parsing a program. For more information see https://en.wikipedia.org/wiki/Abstract_syntax_tree
2. CFG: Control Flow Graph. A graph where basic blocks of a program constitute the nodes and control flow depicts the edges. This data structure is obtained as a result of an optimization pass in a compiler. Most static code analysis require this as a prerequisite. For more information see https://en.wikipedia.org/wiki/Control-flow_graph
3. DFA: Data flow analysis. Data flow analysis sets up recurrence equations the solutions of which can decide if some particular optimization (e.g. Liveness analysis, Code hoisting, Copy propagation, and Common sub-expression elimination) can be done. For more information see https://en.wikipedia.org/wiki/Data-flow_analysis
4. Incorrect use of API: Every API – be it a webservice request, a third-party library call or even the call to a standard library function – has a contract that needs to be followed. Take for example the standard C function strtok:

     char * strtok ( char * str, const char * delimiters );

The contract says: On a first call, the function expects a C string as an argument for str, whose first character is used as the starting location to scan for tokens. In subsequent calls, the function expects a null pointer and uses the position right after the end of the last token as the new starting location for scanning.

Without understanding and following this contract we are guaranteed to have bugs.

5. Operator precedence issues: An expression can involve multiple operands, the precedence of which might not be as intended by the programmer. Take for example the following statement:

if (isUser = AuthenticateUser(username, password) == FAIL) {

The expression involves two operators: the equality operator (==) and the assignment operator (=). Equality operator has higher precedence and we have a classic operator precedence bug. Fixing this uses paranthesis forcing the correct operator precedence :

if ((isUser = AuthenticateUser(username, password)) == FAIL) {

The post What is Static Analysis for C/C++ Code and Why You Should Use It? appeared first on SoftaCheck.