Syntax highlighter header

Monday 15 April 2024

Allowing Client to Client connect on VPN

Recently we were trying to debug an application with two developers one backend developer and one frontend developer. We were working in work from home setup so both were working from home.

We were wasting a lots of effort in pushing backed changes again and again to our staging server for testing. We were thinking that it would be great if frontend developer could connect to service running on backend developer's machine directly and debug the problem. We tried connecting using private IP address of backend developer's machine but connection timed out.

Later on we discovered that there was a configuration needed on OpenVpn server. You need to enable following configuration on OpenVpn server to enable client to client connection. We found this functionality to be of a great help in debugging.

client-to-client

Following post was very useful https://serverfault.com/questions/570316/how-can-multiple-clients-of-an-openvpn-server-find-each-other

Same can be achieved in AWS client VPN also https://docs.aws.amazon.com/vpn/latest/clientvpn-admin/scenario-client-to-client.html

Tuesday 12 March 2024

java.time.ZonedDateTime not supported in ObjectMapper

Recently I was trying to serialize a Java object into json using ObjectMapper but encountered the following exception

com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Java 8 date/time type `java.time.ZonedDateTime` not supported by default: add Module "com.fasterxml.jackson.datatype:jackson-datatype-jsr310" to enable handling

The reason was that Java time module is not supported by default and it does not work just by adding maven dependency. The following code worked for me.

objectMapper = new ObjectMapper();
objectMapper.registerModule(new JavaTimeModule());
objectMapper.disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS);

Thursday 7 March 2024

Finding mongo documents with lowercase values in a mongo collection

Recently we were trying to make our mongo DB based application case insensitive and for testing we needed to find all document containing lowercase alphabets in a filed called "sku". The following query was able to do the work.

{"$expr": {"$ne": ["$sku", {"$toUpper": "$sku"}]}}

Please note that this query does not work on AWS DocumentDB.

 

Wednesday 7 February 2024

Creating a proxy for other service in Spring Boot

Recently we had a requirement to translate from HTTP headers to query parameter to expose a service to a client which can't send query parameter to the service. The client could send the information as HTTP header although.

We decided to build it in our spring boot application using FeignClient. We accepted API name as path variable so that same code can work for multiple APIs.

Following is the code our FeignClient interface which accept API name as a path variable:

import org.springframework.cloud.openfeign.FeignClient;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestParam;

@FeignClient(name = "genericClient", configuration = GenericClientConfiguration.class,
        url = "${base-url}" )
public interface GenericClient {
    String LOGIN_ID = "_loginid";
    String USER_TOKEN = "_token";

    @PostMapping(value = "/prefix/{apiName}", consumes = "application/json", produces = "application/json")
    ResponseEntity<String> callAPI( @PathVariable String apiName,
            @RequestParam(value = LOGIN_ID) String loginId,
            @RequestParam(value = USER_TOKEN) String userToken,
            @RequestBody String requestBody);
}

Following is the code for our controller class which receive the request from client. This controller catches errors returned by the server and passes them to the client.

import feign.FeignException;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;

import java.nio.ByteBuffer;
import java.nio.charset.StandardCharsets;
import java.util.Optional;

@RestController
@Slf4j
@RequestMapping("/api/public")
public class ProxyController {

    @Autowired
    private GenericClient genericClient;

    @PostMapping(value = "/proxyPrefix/{api}", consumes = "application/json", produces = "application/json")
    public ResponseEntity<?> omsApiCall(@PathVariable String api,
                                           @RequestHeader String userName,
                                           @RequestHeader String userToken,
                                           @RequestBody String reqBody) {
        try {
            ResponseEntity<String> resp= genericClient.callAPI(api, userName, userToken, reqBody);
            log.info("Generic call successful API={}",api);
            return resp;
        } catch(FeignException.FeignClientException fce) {
            log.info("Generic call failed: API={}", api, fce);
            Optional<ByteBuffer> respBody = fce.responseBody();
            if(respBody.isPresent()) {
                String errorText = StandardCharsets.UTF_8.decode(respBody.get()).toString();
                return ResponseEntity.status(fce.status()).body(errorText);
            } else {
                return ResponseEntity.status(fce.status()).build();
            }
        } catch(FeignException.FeignServerException fse) {
            log.error("Generic call failed with server error for API={}", api, fse);
            Optional<ByteBuffer> respBody = fse.responseBody();
            if(respBody.isPresent()) {
                String errorText = StandardCharsets.UTF_8.decode(respBody.get()).toString();
                return ResponseEntity.status(fse.status()).body(errorText);
            } else {
                return ResponseEntity.status(fse.status()).build();
            }
        } catch(FeignException fe) {
            log.error("Generic call failed with unknown error for API={}", api, fe);
            Optional<ByteBuffer> respBody = fe.responseBody();
            if(respBody.isPresent()) {
                String errorText = StandardCharsets.UTF_8.decode(respBody.get()).toString();
                return ResponseEntity.status(fe.status()).body(errorText);
            } else {
                return ResponseEntity.status(fe.status()).build();
            }
        }
    }
}

Following is the code for our trusting Feign Client configuration which is needed incase you are connecting to a https url with self signed certificate:

import feign.Client;
import org.apache.http.ssl.SSLContexts;
import org.apache.http.ssl.TrustStrategy;
import org.springframework.context.annotation.Bean;

import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.SSLContext;
import java.security.KeyManagementException;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.security.cert.X509Certificate;

public class GenericClientConfiguration {

    @Bean
    public Client feignClient()
            throws KeyStoreException, NoSuchAlgorithmException, KeyManagementException {
        TrustStrategy acceptingTrustStrategy = (X509Certificate[] chain, String authType) -> true;
        HostnameVerifier hostnameVerifier = (s, sslSession) -> true;
        SSLContext sslContext = SSLContexts.custom().loadTrustMaterial(null, acceptingTrustStrategy).build();
        return new Client.Default(sslContext.getSocketFactory(),hostnameVerifier);
    }
}

Tuesday 12 September 2023

Uploading a large file to S3 from flutter

We were recently trying to upload a large zip file from flutter browser app and we discovered that when we try to upload files larger than 500 MB it was throwing Buffer Allocation error. We needed to upload files larger than 10 GB.

So we started exploring other options. We finally solved the problem using S3 Multipart upload. We did not want to expose S3 credentials to client due to security reasons. So we implemented 3 APIs on server:

  1. InitiateMultipartUpload - This API initiates multipart upload on S3 and return uploadId to client
  2. UploadPart - This API receives a part from client and sends it to S3 and returns PartETag to client
  3. ComplateMultipartUpload - This API completes multipart upload in S3 using PartETags received from client.
Now for client side support we developed a custom component to upload file to S3 taking inspiration from https://github.com/Taskulu/chunked_uploader/blob/master/lib/chunked_uploader.dart

After these changes I was able to upload 5GB file to S3 in 20 minutes without getting buffer allocation error.