小行星

热爱平淡，向往未知

0%

Posted on Views:

git origin是远程主机
可以通过git remove -v 看到

git push origin master的全部含义是把 本地的master分支推送到远程的master

git pull origin master的全部含义同～

Posted on In Views:

可持久化对象不如直接多线程dataloader。

想把处理好的对象直接保存下来，然后写了一个Cache类

第一次会把对象保存到硬盘, 第二次会直接读取

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
import os
import torch
from abc import ABCMeta, abstractmethod


class DataBase(object):
    __metaclass__ = ABCMeta

    def __init__(self):
        pass

    @abstractmethod
    def get_data(self):
        pass


class Cache:
    def __init__(self, path):
        self.path = path

    def fetch(self, name, database):
        save_path = os.path.join(self.path, name)
        if os.path.exists(save_path):
            return torch.load(save_path)
        else:
            data = database.get_data()
            torch.save(data, save_path)
            return data
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
class TrainDataBase(DataBase):
    def get_data(self):
        return [ {'image': data['image'], 'pose': data['pose']}
                for i, data in tqdm(enumerate(train_dataloader))]


class TestDataBase(DataBase):
    def get_data(self):
        return [ {'image': data['image'], 'pose': data['pose']}
                for i, data in tqdm(enumerate(test_dataloader)) ]


persistent_path = "/SSD1/save_obj"
cache = Cache(persistent_path)
time1 = time()

train_data_vec = cache.fetch("mobile-net-train-data", TrainDataBase())
time2 = time()

test_data_vec = cache.fetch("mobile-net-test-data", TestDataBase())
time3 = time()
print ("%s, %s\n", time2 - time1, time3 - time1)

然后测试结果是
74.46734094619751s, 75.0271668434143s
本来多线程是不到1分钟。。。
去生成的文件瞅瞅，12G
因为Dataloader是多线程处理的，所以会快很多。

Posted on In Views:

代码: https://github.com/youngsterxyf/mpdp-code

创建型

1. 工厂模式

场景:
客户端1可以请求一个对象，而无需知道这个对象来自哪里
创建对象的代码和使用对象的代码解耦
我们执行单个函数，传入一个参数(提供信息表明我们想要什么)，但
并不要求知道任何关于对象如何实现以及对象来自哪里的细节。

Read more »

Posted on Edited on Views: 
1
2
3
4
5
6
7
8
9
%load_ext autoreload
%autoreload 2

from foo import some_function
some_function()
42
# open foo.py in an editor and change some_function to return 43
some_function()
43

Posted on Edited on Views:

加载预训练模型

1
2
my_resnet = MyResNet(*args, **kwargs)
my_resnet.load_state_dict(torch.load("my_resnet.pth"))
1
torch.save(my_resnet.state_dict(), "my_resnet.pth")
1
my_resnet = torch.load("my_resnet.pth")

加载部分预训练模型

1
2
3
4
5
6
7
8
pretrained_dict = model_zoo.load_url(model_urls['resnet152'])
model_dict = model.state_dict()
# 将pretrained_dict里不属于model_dict的键剔除掉
pretrained_dict = {k: v for k, v in pretrained_dict.items() if k in model_dict}
# 更新现有的model_dict
model_dict.update(pretrained_dict)
# 加载我们真正需要的state_dict
model.load_state_dict(model_dict)

Posted on Edited on Views: 
1
tar cf - ./images -P | pv -s $(du -sb ./images | awk '{print $1}') | gzip > /disk3/images.tar.gz

Posted on Edited on Views:

Put your cursor on the first # character, press CtrlV (or CtrlQ for gVim), and go down until the last commented line and press x, that will delete all the # characters vertically.

For commenting a block of text is almost the same:

First, go to the first line you want to comment, press CtrlV. This will put the editor in the VISUAL BLOCK mode.
Then using the arrow key and select until the last line
Now press ShiftI, which will put the editor in INSERT mode and then press #. This will add a hash to the first line.
Then press Esc (give it a second), and it will insert a # character on all other selected lines.

https://stackoverflow.com/questions/1676632/whats-a-quick-way-to-comment-uncomment-lines-in-vim

Posted on Edited on Views: 
1
2
pip uninstall ipykernel
pip install ipykernel

Posted on Edited on Views:

使用proxychains的原因

1
PYTHON_CONFIGURE_OPTS="--disable-ipv6" proxychains pyenv install -v 3.6