Recently we were trying to make our mongo DB based application case insensitive and for testing we needed to find all document containing lowercase alphabets in a filed called "sku". The following query was able to do the work.
{"$expr": {"$ne": ["$sku", {"$toUpper": "$sku"}]}}
Please note that this query does not work on AWS DocumentDB.
Recently we had a requirement to translate from HTTP headers to query parameter to expose a service to a client which can't send query parameter to the service. The client could send the information as HTTP header although.
We decided to build it in our spring boot application using FeignClient. We accepted API name as path variable so that same code can work for multiple APIs.
Following is the code our FeignClient interface which accept API name as a path variable:
import org.springframework.cloud.openfeign.FeignClient;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestParam;
@FeignClient(name = "genericClient", configuration = GenericClientConfiguration.class,
url = "${base-url}" )
public interface GenericClient {
String LOGIN_ID = "_loginid";
String USER_TOKEN = "_token";
@PostMapping(value = "/prefix/{apiName}", consumes = "application/json", produces = "application/json")
ResponseEntity<String> callAPI( @PathVariable String apiName,
@RequestParam(value = LOGIN_ID) String loginId,
@RequestParam(value = USER_TOKEN) String userToken,
@RequestBody String requestBody);
}
Following is the code for our controller class which receive the request from client. This controller catches errors returned by the server and passes them to the client.
import feign.FeignException;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;
import java.nio.ByteBuffer;
import java.nio.charset.StandardCharsets;
import java.util.Optional;
@RestController
@Slf4j
@RequestMapping("/api/public")
public class ProxyController {
@Autowired
private GenericClient genericClient;
@PostMapping(value = "/proxyPrefix/{api}", consumes = "application/json", produces = "application/json")
public ResponseEntity<?> omsApiCall(@PathVariable String api,
@RequestHeader String userName,
@RequestHeader String userToken,
@RequestBody String reqBody) {
try {
ResponseEntity<String> resp= genericClient.callAPI(api, userName, userToken, reqBody);
log.info("Generic call successful API={}",api);
return resp;
} catch(FeignException.FeignClientException fce) {
log.info("Generic call failed: API={}", api, fce);
Optional<ByteBuffer> respBody = fce.responseBody();
if(respBody.isPresent()) {
String errorText = StandardCharsets.UTF_8.decode(respBody.get()).toString();
return ResponseEntity.status(fce.status()).body(errorText);
} else {
return ResponseEntity.status(fce.status()).build();
}
} catch(FeignException.FeignServerException fse) {
log.error("Generic call failed with server error for API={}", api, fse);
Optional<ByteBuffer> respBody = fse.responseBody();
if(respBody.isPresent()) {
String errorText = StandardCharsets.UTF_8.decode(respBody.get()).toString();
return ResponseEntity.status(fse.status()).body(errorText);
} else {
return ResponseEntity.status(fse.status()).build();
}
} catch(FeignException fe) {
log.error("Generic call failed with unknown error for API={}", api, fe);
Optional<ByteBuffer> respBody = fe.responseBody();
if(respBody.isPresent()) {
String errorText = StandardCharsets.UTF_8.decode(respBody.get()).toString();
return ResponseEntity.status(fe.status()).body(errorText);
} else {
return ResponseEntity.status(fe.status()).build();
}
}
}
}
Following is the code for our trusting Feign Client configuration which is needed incase you are connecting to a https url with self signed certificate:
import feign.Client;
import org.apache.http.ssl.SSLContexts;
import org.apache.http.ssl.TrustStrategy;
import org.springframework.context.annotation.Bean;
import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.SSLContext;
import java.security.KeyManagementException;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.security.cert.X509Certificate;
public class GenericClientConfiguration {
@Bean
public Client feignClient()
throws KeyStoreException, NoSuchAlgorithmException, KeyManagementException {
TrustStrategy acceptingTrustStrategy = (X509Certificate[] chain, String authType) -> true;
HostnameVerifier hostnameVerifier = (s, sslSession) -> true;
SSLContext sslContext = SSLContexts.custom().loadTrustMaterial(null, acceptingTrustStrategy).build();
return new Client.Default(sslContext.getSocketFactory(),hostnameVerifier);
}
}
We were recently trying to upload a large zip file from flutter browser app and we discovered that when we try to upload files larger than 500 MB it was throwing Buffer Allocation error. We needed to upload files larger than 10 GB.
So we started exploring other options. We finally solved the problem using S3 Multipart upload. We did not want to expose S3 credentials to client due to security reasons. So we implemented 3 APIs on server:
After these changes I was able to upload 5GB file to S3 in 20 minutes without getting buffer allocation error.
Recently were working on uploading a zip file to S3 using presigned url. We exposed an API to client for generating a presigned url over an authenticated API call. After generating presigned URL client is supposed to send content in a PUT call to S3 on the presigned url. But when we tried sending the file using MultipartRequest the file got corrupted. After struggling for a long time we found StreamedRequest using which we were able to upload the file to S3 without getting it corrupted. Following is the code for uploading the file to S3.
Future<http.StreamedResponse?> uploadFileToS3(
String s3Url,
file_picker.PlatformFile file,
) async {
var request = http.StreamedRequest('PUT', Uri.parse(s3Url));
final streamSink = request.sink as StreamSink<List<int>>;
print('Uploading file '+file.name+' to '+s3Url);
var resp = request.send();
await streamSink.addStream(file.readStream!);
streamSink.close();
print('returning response for '+file.name);
return await resp;
}
Please note readStream will be null if you don't pass required arguments to file_picker. The required arguments for file picker are:
FilePickerResult? result = await FilePicker.platform.pickFiles(withData:false, withReadStream: true);
Please let me know if you face any problem in using this approach.
Recently we were required to download a CSV file from our partner website in our Java application. So I explored new Java HttpClient API. The APIs are asynchronous which is good for scalability but confusing for the users. After struggling for a long time I was able to figure out how to download the file which was protected by a username and password.
The code for downloading file is provided below, it also include a call to get just headers using HEAD call.
import java.io.IOException;
import java.net.Authenticator;
import java.net.PasswordAuthentication;
import java.net.URI;
import java.net.URISyntaxException;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.file.Path;
import java.nio.file.StandardOpenOption;
public class HttpClientMainClass {
public static void main(String[] args) throws URISyntaxException, IOException, InterruptedException {
HttpClient httpClient = HttpClient.newBuilder().authenticator(new Authenticator() {
@Override
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication("username","password".toCharArray());
}
})
.build();
HttpRequest httpRequest2 = HttpRequest.newBuilder().method("HEAD", HttpRequest.BodyPublishers.noBody())
.uri(new URI("http://example.com/data.csv"))
.build();
HttpResponse<Void> resp2 = httpClient.send(httpRequest2,
HttpResponse.BodyHandlers.discarding());
System.out.println(resp2);
System.out.println(resp2.headers());
HttpRequest httpRequest = HttpRequest.newBuilder().GET()
.uri(new URI("http://example.com/data.csv"))
.build();
HttpResponse<Path> resp = httpClient.send(httpRequest,
HttpResponse.BodyHandlers.ofFile(Path.of("/tmp","data.csv"),
StandardOpenOption.CREATE, StandardOpenOption.WRITE));
if(resp.statusCode()==200) {
System.out.println("File downloaded At : "+resp.body());
}
}
}
Hope it will be useful for the readers.