Torrent_Suite_Software1. Plugin System2. REST API附录

84 阅读 0 评论 56 点赞

我是靠谱客的博主朴素火，这篇文章主要介绍Torrent_Suite_Software1. Plugin System2. REST API附录，现在分享给大家，希望可以做个参考。

文章目录

1. Plugin System
- 1.1 入门
- - 1.1.1 快速开始
  - 1.1.2 流程概览
- 1.2 参考
- - 1.2.1 Base class
  - 1.2.2 运行环境
  - 1.2.3 配置
  - 1.2.4 输入文件
  - - barcodes.json
    - startplugin.json
    - Seq Files (BAMs)
  - 1.2.5 输出文件
- 1.3 示例
- - 1.3.1 basic
  - 1.3.2 API
2. REST API
- 2.1 入门
- - 认证
  - 分页
  - 排序、过滤
- 2.2 参考
- 2.3 示例
- - 2.3.1 Fetching all Chips
  - 2.3.2 Adding Filters
附录
- Generate HTML by templates

Notes from Torrent SDK Documentation, Release 5.12.1.

1. Plugin System

1.1 入门

插件是通用分析流程的扩展，表现为在流程不同环节执行的python模块。

1.1.1 快速开始

'''
vi Myplugin/MyPlugin.py # 1.name; 2.code below
zip -r  –exclude=*.git* mp.zip MyPlugin
upload, run
'''
import subprocess
from ion.plugin import *

class MyPlugin(IonPlugin):
# Must inherit from the base class IonPlugin;
# The version attribute and launch method need to be overridden;
    
    version = "1.0.0.0"

    def launch(self, data=None):
        output = subprocess.check_output(['ls','-l'])
        with open("status_block.html","w") as h:
            h.write("<html><body><pre>")
            h.write(output)
            h.write("</pre></body></html>")

if __name__ == "__main__":
    PluginCLI()

1.1.2 流程概览

Run levels	stage
PRE	before any significant processing happens
BLOCK	triggered once per block on the chip
POST	after the analysis pipeline is completely executed
DEFAULT	triggered at the end of pipeline processing after the four usual steps
LAST	executed after all other plugins that are not “LAST” have been executed

1.2 参考

1.2.1 Base class

Attribute Name	Required	Type	Default	Description
name	N	str	Empty	存放插件名
version	Y	str	Empty	版本号
runtypes	N	list(str)	Empty	用于整张芯片或缩略
runlevels	Y	list(str)	Empty	运行等级
features	N	list(str)	Empty	功能特性
depends	N	list(str)	Empty	依赖的插件
major_block	N	boolean	False	插件结果是否作为运行报告的一部分
startplugin	N	dictionary	None	返回startplugin.json内存字典中的 “an”

Method Name	Required	Return Type	Description
launch	Y	boolean	插件结果是否作为运行报告的一部分
pre_launch	N	boolean	是否在launch前运行
barcodetable_columns	N	list	generate the plugin barcodes table ui
barcodetable_data	N	dictionary	populate the contents of the plugin barcodes table ui
get_restobj	N	dictionary	based on a REST API function call

1.2.2 运行环境

ion_plugin_*<Plugin Name>*_launch.sh
- 插件执行的入口
- 更新数据库状态
- 设置环境变量
- 设置umask为0000
运行层级（详见1.1.2）
- block
- conventional
集群和网络
- head节点不运行插件
- 通过NFS分发代码到计算节点：/results/plugins/
依赖
- 依赖被直接打包到插件的文件夹（/results/plugins/<Plugin Name> ）进行分发
REST API
- 通过在插件根目录下 extend.py 中实现一个bucket字典参数的方法，就能通过REST API暴露扩展的自定义端点。

# http(s)://{HOSTNAME}/rundb/api/v1/plugin/{PLUGIN}/extend/{METHOD_NAME}

1.2.3 配置

global（ MyPlugin/config.html; /rundb/api/v1/plugin/<TB_plugin.pk>/）
plan（MyPlugin/plan.html）
manual（MyPlugin/instance.html）
Barcode Table UI：用于选取barcode，指定参数的GUI。

# plugin class method to specify which columns to display
def barcodetable_columns(self):
    cols=[
    {"field": "selected", "editable": True},
    {"field": "barcode_name", "editable":True},
    {"field": "sample", "editable": False}
    ]
    return cols
    
# plugin class method to specify default table contents
def barcodetable_data(self, data, planconfig={}, globalconfig={}):
    ## data: same structure as in barcodes.json
    ## planconfig: plugin configuration from planning (plan.html)
    ## globalconfig:plugin global configuration (config.html)
    return data

The instance.html must add the contents of barcodes table to the plugin data before POSTing it to the results API.
This data will be written to startplugin.json file at plugin runtime under “pluginconfig” section.

1.2.4 输入文件

barcodes.json

nonbarcoded

keys	type	describe
aligned	bool	标识bam_file中的reads是否已比对到参考基因组
bam_file	str	存储reads的文件名
bam_filepath	str	bam_file在服务器上的路径
control_sequence_type	str	（RNA测序）ERCC Mix 1 or ERCC Mix2
filtered	bool	标识barcode是否通过了TS分析流程过滤
hotspot_filepath	str	热点目标区域文件（BED格式）在服务器上的路径
genome_urlpath	str	基因组的URL地址
nucleotide_type	str	DNA或RNA
read_count	int	barcode在bam_file中的总数
reference	str	参考基因组的常用名，如hg19
reference_fullpath	str	参考序列的全路径
sample	str	和barcode关联的样本名
sample_id	str	和sample关联的样本ID
target_region_filepath	str	靶向区域文件（BED格式）的全路径

barcode_name - nonbarcoded

keys	type	describe
barcode_adapter	str	从测序read中区分barcode_sequence的接头序列
barcode_annotation	str	用户对barcode的注释
barcode_index	int	barcode在barcode集中的索引
barcode_name	str	barcode在barcode集中的名字
barcode_sequence	str	用于标识barcode的DNA序列
barcode_type	str	用户指定的类型

startplugin.json

keys	type	describe
chefSummaary	dict	chef parameters used

keys	type	describe
datamanagement	dict	data management state
__Basecalling Input	bool
__Intermediate Files	bool
__Output Files	bool
__Signal Processing Input	bool

keys	type	describe
expmeta	dict	aggregate from the expMeta.dat file and the ion_params_00.json file
__analysis_date	date
__barcodeId	str
__chipBarcode	str
__chiptype
__flowOrder
__instrument
__notes
__project
__results_name
__run_date
__run_flows
__run_name
__runid
__sample
__output_file_name_stem

keys	type	describe
globalconfig
__MEM_MAX
__debug

keys	type	describe
plan
__barcodeId
__barcodedSamples
____Sample Name
______-Sample Name-
________barcodeSampleInfo
__________-Barcode ID-
____________controlSequenceType
____________controlType
____________description
____________externalId
____________hotSpotRegionBedFile
____________nucleotideType
____________reference
____________sseBedFile
____________targetRegionBedFile
________barcodes
__bedfile
__controlSequencekitname
__librarykitname
__planName
__regionfile
__reverse_primer
__runMode
__runType
__runTypeDescription
__sampleGrouping
__samplePrepKitName
__sampleSet_name
__sampleSet_planIndex
__sampleSet_planTotal
__sampleSet_uid
__sampleTubeLabel
__sequencekitname
__templatingKitName
__threePrimeAdapter
__username

keys	type	describe
runinfo	dict	测序run的信息
__alignment_dir	str	比对数据的文件夹路径
__analysis_dir	str
__api_key	str	用于访问
__api_url
__barcodeId
__basecaller_dir
__chipDescription
__chipType
__library
__library_key
__net_location
__pk
__platform
__plugin	dict	the run parameters for this plugin in this run
____depends	list	该插件本次运行所依赖的插件
____features	list	该插件的功能
____hold_jid	list	处理所需的 job ID
____id	int	插件ID的数据库主键
____name	str	该插件的名字
____path	str	该插件可执行文件的路径
____pluginconfig	dict	插件运行的全局配置信息
____pluginresult	int	插件结果entry的数据库主键
____results_dir	str	插件分析结果的目录路径
____runlevel	list	运行层次
____runtype	list	运行类型
____userInput	dict	插件运行的配置信息
____version	str	插件版本
__plugin_dir
__plugin_name
__pluginresult
__raw_data_dir
__report_root_dir
__results_dir
__sigproc_dir
__systemType
__testfrag_key
__tmap_version
__url_root
__username
____runplugin
______blockId
______block_dirs
______numBlocks
______run_mode
______run_type
____runlevel
__sampleinfo
____SampleName
______attributes
______description
______displayedName
______externalId
______name
____pluginconfig

Seq Files (BAMs)

barcodes.json >> bam_filepath

1.2.5 输出文件

The report HTML file (*_block.(html|php))
A log file (drmaa_stdout.txt)

1.3 示例

1.3.1 basic

# LogParser
import json,os
from django.utils.functional import cached_property
from ion.plugin import *

class logParser(IonPlugin):
    version = "0.1.0.0"
    
    @cached_property
    def startplugin_json(self):
        return self.startplugin
        
    def read_explog(self):
        path = os.path.join(self.startplugin_json['runinfo']['raw_data_dir'],'explog_final.txt')
        if not os.path.exists(path):
            raise Exception("explog_final.txt not found")
        md = {}
        for l in open(path):
            ll = l.split(":",1)
            if len(ll) == 2:
                k,v = ll
                md[k.strip()] = v.strip()
        return md
        
def launch(self, data=None):
    na = '<strong>NA</strong>'
    explog = self.read_explog()
    result = {'Sample':explog.get('Sample',None) or na}
    with open(self.startplugin_json['runinfo']['results_dir']+'/LogParser_block.html','w') as ha:
        ha.write('<html><body>')
        ha.write("Sample is %s<br >" % result['Sample'])
        ha.write("</body></hmtl>")
    with open(self.startplugin_json['runinfo']['results_dir'] + '/results.json','w') as ha:
        json.dump(result, ha, indent=4)
    return True

if __name__ == "__main__":
    PluginCLI()

# subprocess
from subprocess import check_output
import json
import os
from django.utils.functional import cached_property
from ion.plugin import *

class CallSubprocessExample(IonPlugin):
    version = "5.4.0.0"
    runtypes = [RunType.FULLCHIP, RunType.THUMB, RunType.COMPOSITE]
    runlevels = [RunLevel.DEFAULT]

    @cached_property
    def startplugin_json(self):
        return self.startplugin

    def launch(self, data=None):
    """This is the primary launch method for the plugin."""
        path_to_executable = "MyExecutable"
        arg1 = 'First Argument'
        arg2 = 'Second Argument'
        results = check_output([path_to_executable, arg1, arg2],cwd=self.startplugin_json['runinfo']['plugin']['path'])
        return True

if __name__ == "__main__":
    PluginCLI()

# access barcode data
import json
import os
from django.utils.functional import cached_property
from ion.plugin import *
from subprocess import check_output

class BarcodesExample(IonPlugin):
    version = "5.4.0.0"
    runtypes = [RunType.FULLCHIP, RunType.THUMB, RunType.COMPOSITE]
    runlevels = [RunLevel.DEFAULT]

    @cached_property
    def startplugin_json(self):
        return self.startplugin

    @cached_property
    def barcodes_json:
        with open('barcodes.json', 'r') as barcodes_handle:
    return json.load(barcodes_handle)

    def launch(self, data=None):
    """This is the primary launch method for the plugin."""
    for barcode_name, barcode_values in self.barcodes_json.iteritems():
        print("Barcode Name: " + barcode_name)
        print("Bam File: " + barcode_values['bam_file']
    return True

if __name__ == "__main__":
PluginCLI()

1.3.2 API

# use REST API
import json
import requests
api_response = requests.get('http://HOSTNAME/APPNAME/api/v1/APIENDPOINT/?ARG1=VAL1&pluginresult=self.startplugin['runinfo']['pluginresult']&api_key=' + self.startplugin['runinfo']['api_key'])
api_response.raise_for_status()
api_json_response = json.loads(api_response.content)

# get the installed annotation files
import os
import sys
import requests
import json
from ion.plugin import RunType, PluginCLI, IonPlugin

API_VER = '/v1'

class PluginApiTest(IonPlugin):
    version = "1.0"
    runtypes = [RunType.THUMB, RunType.FULLCHIP]

def launch(self):
    sp_json_fpath = os.path.join(os.environ.get('RESULTS_DIR', ''),'startplugin.json')
    with open(sp_json_fpath, 'r') as sp_fh:
        sp_json = json.load(sp_fh)
        endpoint = '/content'
        auth_params = {'api_key': sp_json['runinfo']['api_key'],
                       'pluginresult': sp_json['runinfo']['pluginresult'],}
        query_params = {'type': 'Annotation'}
        request_params = {}
        request_params.update(auth_params)
        request_params.update(query_params)
        url = sp_json['runinfo']['api_url'] + API_VER + endpoint
        resp = requests.get(url, params=request_params)
        if resp:
            print(resp.url)
            print(resp.json())
            sys.exit(0)
        else:
            print(resp.status_code)
            sys.exit(1)
            
if __name__ == "__main__":
        PluginCLI()

2. REST API

所有资源使用HTTP标准动词返回json数据。

End Point	return
/rundb/api/v1/	包含可获取资源信息的JSON对象
~/resource-name/	资源列表
~/resource-name/schema/

2.1 入门

认证

在 /admin/tastypie/apikey/ 下查看用户的 API keys。以 header 或 GET params 的方式在请求中使用 API key。

# header
Authorization: ApiKey <username>:<api_key>
# GET params
http://127.0.0.1:8000/api/v1/entries/?username=<username>&api_key=<api_key>

分页

Response 的 meta 包含分页信息。以 GET params 的方式控制分页。

field	describe
limit	可包含的最大资源数量
next	指向下一页的URL
offset	当前对象列表开始的对象编号
previous	指向上一页的URL
total_count	过滤后留下的对象总数

排序、过滤

使用 order_by GET params 指定排序字段。
使用 name=<filed_name> GET params 完成对指定字段的基本过滤。

2.2 参考

Active Ion Chef Library Prep Kit Info Resource
Active Ion Chef Prep Kit Info Resource
Active Library Kit Info Resource
Active Pgm Library Kit Info Resource
Active Pgm Sequencing Kit Info Resource
Active Proton Library Kit Info Resource
Active Proton Sequencing Kit Info Resource
Active Sequencing Kit Info Resource
Analysis Args Resource
Analysis Metrics Resource
Application Group Resource
Appl Product Resource
Available Ion Chef Planned Experiment Resource
Available Ion Chef Planned Experiment Summary Resource
Available Onetouch Planned Experiment Resource
Available Onetouch Planned Experiment Summary Resource
Available Planned Experiment Summary Resource
Chip Resource
Cluster Info History Resource
Common Cv Resource
Composite Data Management Resource
Composite Experiment Resource
Composite Result Resource
Content Resource
Content Upload Resource
Data Management History Resource
Dna Barcode Resource
Email Address Resource
Event Log Resource
Experiment Resource
Experiment Analysis Settings Resource
File Monitor Resource
File Server Resource
Flow Order Resource
Get Chef Cartridgeusage Resource
Get Chef Script Info Resource
Global Config Resource
Ion Chef Plan Template Resource
Ion Chef Plan Template Summary Resource
Ion Chef Prep Kit Info Resource
Ion Mesh Node Resource
Kit Info Resource
Kit Part Resource
Lib Metrics Resource
Library Key Resource
Library Kit Info Resource
Library Kit Part Resource
Location Resource
Log Resource
Message Resource
Monitor Data Resource
Monitor Result Resource
Network Resource
Onetouch Plan Template Resource
Onetouch Plan Template Summary Resource
Planned Experiment Resource
Planned Experiment Db Resource
Planned Experiment Qc Resource
Plan Template Basic Info Resource
Plan Template Summary Resource
Plugin Resource
Plugin Result Resource
Plugin Result Job Resource
Project Resource
Project Results Resource
Publisher Resource
Qc Type Resource
Quality Metrics Resource
Reference Genome Resource
Results Resource
Rig Resource
Run Type Resource
Sample Resource
Sample Annotation Cv Resource
Sample Attribute Resource
Sample Attribute Data Type Resource
Sample Group Type Cv Resource
Sample Prep Data Resource
Sample Set Resource
Sample Set Item Resource
Sample Set Item Info Resource
Sequencing Kit Info Resource
Sequencing Kit Part Resource
Support Upload Resource
Template Resource
Tf Metrics Resource
Three Prime Adapter Resource
Torrent Suite Resource
User Resource

2.3 示例

2.3.1 Fetching all Chips

import requests
BASE_URL = "http://example.xyz"
USERNAME = "ionadmin"
API_KEY = "efb7a14021732d773a4258b69d9452042a31a6b6"

# Chip Resource API
headers = {"Authorization": "ApiKey " + USERNAME + ":" + API_KEY}
object_list = []
next_url = "/rundb/api/v1/chip/"
while next_url:
    response = requests.get(BASE_URL + next_url, headers=headers, params={})
    response.raise_for_status()
    response_data = response.json()
    object_list += response_data["objects"]
    next_url = response_data["meta"]["next"] or None
print object_list

2.3.2 Adding Filters

#  Chip Resource API
headers = {"Authorization": "ApiKey " + USERNAME + ":" + API_KEY}
object_list = []
next_url = "/rundb/api/v1/chip/"
while next_url:
    response = requests.get(BASE_URL + next_url, headers=headers, params={"name__startswith": "31"})
    response.raise_for_status()
    response_data = response.json()
    object_list += response_data["objects"]
    next_url = response_data["meta"]["next"] or None
for chip in object_list[0:3]:
    print chip["name"]

附录

Generate HTML by templates

Templates

最后

以上就是朴素火最近收集整理的关于Torrent_Suite_Software1. Plugin System2. REST API附录的全部内容，更多相关Torrent_Suite_Software1.内容请搜索靠谱客的其他文章。

本图文内容来源于网友提供，作为学习参考使用，或来自网络收集整理，版权属于原作者所有。

本文分类：生物信息
浏览次数：84 次浏览
发布日期：2023-09-05 21:15:08
本文链接：https://www.kaopuke.com/article/k-p-k_14_uzo_10_f1_13_j_2_0.html

Torrent_Suite_Software1. Plugin System2. REST API附录

文章目录

1. Plugin System

1.1 入门

1.1.1 快速开始

1.1.2 流程概览

1.2 参考

1.2.1 Base class

1.2.2 运行环境

1.2.3 配置

1.2.4 输入文件

barcodes.json

startplugin.json

Seq Files (BAMs)

1.2.5 输出文件

1.3 示例

1.3.1 basic

1.3.2 API

2. REST API

2.1 入门

认证

分页

排序、过滤

2.2 参考

2.3 示例

2.3.1 Fetching all Chips

2.3.2 Adding Filters

附录

Generate HTML by templates

最后

评论列表共有 0 条评论

发表评论取消回复

Torrent_Suite_Software1. Plugin System2. REST API附录

文章目录

1. Plugin System

1.1 入门

1.1.1 快速开始

1.1.2 流程概览

1.2 参考

1.2.1 Base class

1.2.2 运行环境

1.2.3 配置

1.2.4 输入文件

barcodes.json

startplugin.json

Seq Files (BAMs)

1.2.5 输出文件

1.3 示例

1.3.1 basic

1.3.2 API

2. REST API

2.1 入门

认证

分页

排序、过滤

2.2 参考

2.3 示例

2.3.1 Fetching all Chips

2.3.2 Adding Filters

附录

Generate HTML by templates

最后

相关文章

评论列表共有 0 条评论

发表评论 取消回复

发表评论取消回复