Read the docs

These days, I finished the document for the MediCurator.


Below is the link:

http://medicurator.readthedocs.io/en/latest/

Document and fix the bug

This week, I wrote the document and fix the bug.

Details at

Download only what is new

https://bitbucket.org/BMI/medicurator/issues/6/download-only-what-is-new-when-attempting


The document:

https://bitbucket.org/BMI/medicurator/issues/5/medicurator-readthedocs-documentation

Fix the hard code the API_key

https://bitbucket.org/BMI/medicurator/issues/7/avoid-the-need-to-hard-code-the-api-key

Research on Medicurator Supporting Dicomweb

I have done some research on the dicomweb. It can apply to MediCurator. The dicomweb has three levels - study, series and instance. I can query it level by level and inherit it to Medicurator like TCIA. The user only need to add the Url of the server, it can work. And I can use the function of retrieve to download the images. So the Madicurator can implement via it. However, I only find a serverhttp://www.dicomserver.co.uk/DICOMRS.html on which the dicomweb has implemented while the retrieve function didn't finished. So I can not experiment with retrieve function. In conclusion, Medicurator can use the dicomweb without too much modification.

MediCurator Refactor and Add the Duplicate Detect

This week, I refactor the MediCurator into the three modules,

1. medicurator-core for the core medicurator stuff.
2. medicurator-server for the API.
3. medicurator-client for the web app.

So that , I can avoid the dependencies conflicts.

And I add the near Duplicate Detect function to the web app and APIs.

    http://localhost:4567/duplicateSets?replicasetID1=***&replicasetID2=***

This makes the functions completely.

What's more, I write some scripts which make user to run my project easier.


Building

       ./compile.sh


Run webapp

       ./run_servlet.sh


Run Restful API

        ./run_api.sh

Restful API and Delete - Download Workflow

Restful API and Delete - Download Workflow

First, this week, I implement the Restful API.

As shown in the README, it concludes the following

API:

http://localhost:4567/signup?username=***&password=***

http://localhost:4567/login?username=***&password=***

http://localhost:4567/getReplicaSets?userid=***

http://localhost:4567/createReplicaSets?userid=***&replicaName=***

http://localhost:4567/getDataSets?replicasetID=***

http://localhost:4567/addDataSet?replicasetID=***&datasetID=***

http://localhost:4567/removeDataSet?replicasetID=***&datasetID=***

http://localhost:4567/getRootDataSets http://localhost:4567/getSubsets?datasetID=***

http://localhost:4567/downloadDataSets?datasetID=***


http://localhost:4567/downloadOneDataSets?datasetID=***

http://localhost:4567/deleteDataSets?datasetID=***

http://localhost:4567/deleteOneDataSet?datasetID=***

And there is

http://localhost:4567/duplicateSets?replicasetID1=***&replicasetID2=***

This is to be done.

The second thing I have done is fix the Delete - Download Workflow. Now Medicurator can support the function that the user download and delete and download again.

I implement this by taking the meaning of "remove" and "delete" apart. Remove means to move the dataset out of the replicaset and "delete" means delete directly which can download again.

Implement Local File Source

       This week,  I implement the medicurator  for Local file source.

       Through my test, it proves to have been working well. The test  local file is under the path:medicurator/target/classes/image

       It can be downloaded through the web application written last week.

      The downloaded file is now stored at medicurator/target/classes/local.test
      The path can be changed in Constant.java
   
      Afterwards, I will finish the complete workflow on downloaded tracking to solve the delete problem. I think I should implement a delete invoke function so that when the user's delete behavior  through the website or the delete message sent from the duplicate detect, it will invoke the function.

 

Hdfs Apply to Medicurator

This week, I apply Hdfs to Medicurator.

As we all know, the hadoop distributed file system(HDFS) is a distributed file system designed to run on commodity hardware. HDFS is highly fault-tolerant and is designed to be deploved on low-cost hardware. Hdfs provides high throughput access to application data and is suitable for applications that have large data sets.

To make medicurator easier to deal with the high throughput, I decide to use the Hdfs. I inherit the class Storage and make the already existed LocalStorage become HdfsStorage. In order to realize this, I mainly use the API, referred https://hadoop.apache.org/docs/r2.6.1/api/overview-summary.html.

After my test, it works well. I only run this on my single computer, to make this run on the cluster, there still has some work to do.

To add, the consumer can choose the localStorage or the hdfsStorage according to their peference by changing the STORAGE (hdfs/local) in Constants.java. To use HDFS, the user should config HDFS_URI and HDFS_BASEDIR in Constants.java.For example, HDFS_URI = "hdfs://localhost:9000/" and HDFS_BASEDIR = "/user/xxx/medicurator/"
Source code and More information
https://bitbucket.org/BMI/medicurator