Applications

This section explains the concepts of Applications and how to create an application from scratch. Applications serve as development environments where users can design, develop, and test custom logic or transformations before publishing them as Custom Transforms.

Key Differences Between Application and Custom Transform

Aspect	Application	Custom Transform
Usage	Design, develop, and test custom logic	Perform transformations in data pipelines
Pipeline Integration	Cannot be added to pipelines	Can be added to pipelines

To learn more about Custom Tansforms, please read: Custom Transforms

Getting Started

This section explains concepts of Applications and how to create an application from scratch.

Connect with Version Control

Integrate your Git repository (e.g., GitHub, GitLab, Bitbucket) with Condense for seamless code management. You can connect your Condense account to your chosen Git provider. Condense offers Github, Gitlab and Bitbucket as version control.

How to connect the Version Control

1. Go to Settings > Version Control.

1. Connect your Condense account to your chosen Git provider.

Development Environment Expiry and Deletion Notification System

It is designed to manage the lifecycle of the development environment used during the application or custom transform build process. This system ensures that unused or expired containers do not consume unnecessary resources by notifying users of upcoming expirations and automatically deleting containers that meet specific criteria.

Step 1: Go to Applications > Add New Application.

Step 2: Development Environment Initialization:

• The moment you click Add New Application, the development environment is initialized and deployed.

• An Application is created, which can be used to develop and test custom logic or transformations.

Step 3: Set Expiry Threshold:

• After successful deployment, select the threshold for the development environment expiry.

• By default, the system provides 3 days as the expiry timeframe.

Step 4: Click Next to proceed.

Application Lifecycle

Application Status

Draft Status

An application that is not yet published is in Draft Status. It is still under development and can be edited or deleted.

Published Status

Once an application is finalized and published, it moves to Published Status. A published application represents a finalized version of the custom transform and can be used in data pipelines.

Build Application using an IDE with Condense UI

Condense provides a built-in code editor where you can write the logic for your custom transform. You can select branch and repo and manage your code versions through an integrated version control system. You can develop, debug, and build custom transforms using an IDE with Condense.

Select the Branch and Repository

Step 1: Choose your Git Provider Select the Git Provider to build a custom transform

Step 2: Select Repository and Branch: Choose the repository and branch you want to work on

Write and Edit Code

1. Open the Code Editor Once the branch and repo are selected, open the code editor.

2. Write Code

   Write the logic for your custom transform. Use the terminal in the Condense UI to install dependencies. Run commands to install necessary libraries and tools.

3. Use Assisted Coding

   Utilize prompts to generate code snippets for faster development.

4. Edit Code

   Make changes to your code as needed.

Debug the Code

Run Debugger

Start the debugger to execute the code step-by-step. Inspect variable values and identify issues during debugging. Make necessary changes to the code based on debugging insights.

Commit and Push Changes

Commit Changes:

Save your changes locally by initiating a commit. As you write, edit, and modify your code within the Condense code editor, your local development environment tracks those changes. When you are ready to save a specific point in your code's development, you initiate a commit.

Push Changes:

Push your changes to the remote repository. Once you have developed and tested your custom code within the Condense code editor, you can initiate a push operation. This action securely transmits your local code changes to the remote repository hosted on your chosen VCS platform (e.g., GitHub).3.4 Build and Publish the Custom Transform as an Application.

Build and Publish the Application as a custom Transform

Build and publish your custom transform as a deployable application.

Build Application

Click Publish Application and provide the name of the image and tag value to create a build

Review Build Logs:

Check the build logs for commands, outputs, errors, dependencies, and build status. Build logs refer to the record of events generated during the process of building your custom code into a deployable container image. These logs provide valuable information for troubleshooting any issues that might arise during the build stage.

Here's a breakdown of what build logs contain:

• Commands: The sequence of commands executed during the build process, including dependency installation.

• Outputs: The output generated by each command, which can provide information about successful execution or potential errors.

• Errors: Any errors or warnings encountered during the build process are clearly displayed in the logs. This helps you identify and fix issues that might prevent a successful build.

• Dependencies: Information about the installation and configuration of any libraries or tools required for your code to function.

• Build Status: The overall success or failure of the build process is indicated within the logs.

Configure env variables

This section allows users to specify parameters such as ports and environment variables for deploying application containers. These configurations are utilized during the deployment of applications on pipelines

Here's a breakdown of what this contains:

API Server Expose Option

Users are given the choice to expose the API server to external systems. This enables communication between the container and other services or clients. It allows access to the application via the specified port and auto-generated URL.

1. If the user selects Yes, specifying a port number becomes mandatory.

2. If the user selects No, no port configuration is required.

Env Variables Users can configure env variables in this section. Below is an example of environment variables for a Data Processing Application. Specify the env variable, type and description.

• DB_HOST:

  • Type: String

  • Description: The host address of the database

• DB_PORT:

  • Type: Integer

  • Description: The port number for the database connection.

• DB_USER:

  • Type: String

  • Description: The username for database access.

• DB_PASSWORD:

  • Type: String

  • Description: The password for database access. Even though this is a password, it is stored as a string.

• DB_NAME:

  • Type: String

  • Description: The name of the database where processed data will be stored.

• Save the configurations.

Publish the Application as a Custom Transform

Name your custom transform to be used as an Application Name for configuration.

Edit Application

The Edit Application feature allows users to modify applications that are in Draft Status. These applications are still under development and can be edited to refine custom logic, configurations, or code before publishing them as Custom Transforms.

Step 1: Access the Applications Page

a. Navigate to the Applications Section:

Go to the Applications page.

b. Locate the Draft Application:

Find the application card under In Drafts status that you want to edit.

Step 2: Enter Edit Mode

a. Click on the Draft Application Card:

Select the draft application you wish to edit.

b. Spawn Development Environment:

The development environment associated with the draft application will be spawned, allowing you to make changes.

Step 3: Make Changes

a. Edit Code:

Open the code editor and modify the logic or transformations as needed.

b. Test Changes:

Use the debugging tools to verify custom logic.

c. Update Configurations:

Adjust environment variables, ports, or other settings.

Step 4: Update Development Environment Expiry

Update Development Environment Expiry:

The development environment expiry is updated when you access the draft application, ensuring the container remains active for the specified timeframe.

Step 5: Resume from Previous Stage (if applicable)

Resume from Build Stage:

• If a build has already been created for the application, you can resume your journey directly from that stage.

• This allows you to continue working without starting from scratch.

IMPORTANT! Draft Status: Applications in Draft Status are editable and can be modified until they are published.

Development Environment Expiry: Accessing a draft application updates the development environment expiry, ensuring the container remains active for the specified timeframe.

Build Stage: If a build has already been created, you can resume from that stage, saving time and effort.

A published application can be edited directly if its development environment is still active. If the environment has expired, use the Restore option to create a new version for editing.

Delete Application

Provides users with the ability to remove applications either in “Published” status or “in drafts” status that are no longer needed, helping to manage and declutter the workspace.

1. Select the Application:

   • Navigate to the Applications section.

   • Choose the application that is in the "In Draft" status or “Published” status

2. Delete the Draft/Published Application:

   • Click on the Delete option.

   • Confirm the deletion when prompted.

IMPORTANT! Deleting a draft application will remove it entirely from the workspace. No published Custom Transform Versions or Deployments will be affected.