The future of ARC projects

In the discussion “Redesign data storage - removal of Saved requests”, I proposed ending of “saved” requests in the Advanced REST Client, moving all existing “saved” requests to a “default” project, and redesigning what a project is. I was unclear how to evolve this functionality back then, but today the path is clear. This post explains the way forward for ARC projects and the invitation for open discussion before finalizing the spec and turning it into a code.

History of projects

Advanced REST Client (ARC) projects are simple “Saved” requests lists. Saved means they are stored the same way as the “history”, but the application allows defining additional metadata like name or description. A project conceptually is an object with a name and description with zero or more saved requests. The purpose of having a project is to group “saved” requests and organize them in the application. Otherwise, it serves no purpose.

Projects as functional lists

After reading comments in the post referenced at the top of this post and after having hours discussing what an API project means, I decided to change ARC’s projects. I’ll explain where I plan to go with ARC projects in the next paragraph. Note that this is an intermediate step to add support for API design in ARC. To ensure backward compatibility, I plan to implement changes to the ARC project that is the groundwork for future development of the application targeting API design.

We base the new concept of a Project in ARC on correlating HTTP requests for the same API into a single structure. You will be able to make “folders”, which are a way to group HTTP request into even more defined lists. Each folder and the project can have a list of environments defined on each level. A folder inherits environments from a parent. However, a folder may set a flag to limit this inheritance. An environment is a definition of a server, variables, and authentication. This way, you can switch environments (variables and server definition) to make HTTP requests to different physical environments without changing any of the HTTP requests definitions.

A “Project Runner” will allow you to run requests defined in a project or a folder one-by-one and render the execution log in the UI (CLI, the application).

Finally, a project is a formal definition used in ARC. It is a well-defined standard and encapsulated, meaning all information is included in the project. The base format for a project is JSON. However, through the AMF parser, we plan to add support for the YAML format.

Design principles

We design projects to be accessible, standard, and usable. 

Accessible as we build the libraries used in ARC as NodeJS libraries. You can use these libraries to make your integrations on top of them. From these libraries, you can manipulate the project and execute HTTP requests. On top of the libraries, we plan to build a CLI that performs a set of designed operations to manage and execute a project. Finally, we will use the same libraries in ARC as a functional layer under the UI.

Standard as we define data using industry standards (formats) or provide interfaces to exchange ARC formats to other standards. This way, you can switch from ARC to other tools and back.

Usable, we define it as a connected experience. Operations in the UI can also be done in the CLI and with the libraries.

Use cases for ARC projects

API development

During the API implementation, a developer creates a project for the API. The developer creates a folder with HTTP requests that perform authentication and set variables with the access token. They configure a server with the base URI and use reserved variables in the HTTP request definition to access the value. Next, the developer creates a folder with a list of requests related to manipulating a resource in the API. They define a set of CRUD operations. 

During the development, they execute the requests in a folder to ensure the changes they make in the code do not influence the built application’s functional layer.

Naive testing

The developer executes the request in a folder in the CI/CD pipeline as unit tests. They use NodeJS libraries or a CLI to integrate the testing with their custom pipeline. We call it naive testing as even though these tests can function as a test layer for an API, they cannot replace structured testing of an API in the long run. 

Current projects and “saved” requests

Given the discussion we had in the referenced issue report, we plan to end-of-life of the current projects and “saved” requests. All currently “saved” requests that do not belong to a project we will move to a “default” (or similar) project.

Beyond the current scope of changes

After introducing the new projects in ARC (we target v18, H1 2022), we plan to work on structural API testing. We are missing details, but we plan to design a testing language for APIs, which the libraries translate to the code at runtime. This way, we design tests like we design APIs and the runtime libraries take care of the execution of the test. We believe it is a highly scalable way of testing APIs as this allows us to design once and run in different environments.

Please, let us know what you think. We are happy to hear your perspective on this. Before implementing the new projects, we can still shape the whole concept together. You can join the discussion in this GitHub post: https://github.com/advanced-rest-client/arc-electron/discussions/434

Regards

Pawel Uchida-Psztyc