CuriousY A world with wonder

Play Python Library之pytest--xunit-style setup篇

| Comment

Is ‘xunit-style setup’ a fixture?

关于什么是’xunit-style setup’请看官方文档

之所以觉得’xunit-style setup’是pytest fixture,是因为官方文档上有提到:

This section describes a classic and popular way how you can implement fixtures (setup and teardown test state) on a per-module/class/function basis.

先看pytest的代码中’xunit-style setup’是如何实现的:

class Class(PyCollector):
    """ Collector for test methods. """
    def collect(self):
        if hasinit(self.obj):
            self.warn("C1", "cannot collect test class %r because it has a "
                "__init__ constructor" % self.obj.__name__)
            return []
        elif hasnew(self.obj):
            self.warn("C1", "cannot collect test class %r because it has a "
                            "__new__ constructor" % self.obj.__name__)
            return []
        return [self._getcustomclass("Instance")(name="()", parent=self)]

    def setup(self):
        setup_class = _get_xunit_func(self.obj, 'setup_class')
        if setup_class is not None:
            setup_class = getattr(setup_class, 'im_func', setup_class)
            setup_class = getattr(setup_class, '__func__', setup_class)
            setup_class(self.obj)

        fin_class = getattr(self.obj, 'teardown_class', None)
        if fin_class is not None:
            fin_class = getattr(fin_class, 'im_func', fin_class)
            fin_class = getattr(fin_class, '__func__', fin_class)
            self.addfinalizer(lambda: fin_class(self.obj))

上面代码是setup_classteardown_class的实现代码。可以看到虽然它的实现和fixture有点像,但肯定不是一个fixture(不光它们的类型不同,调用它们的pytest hook也不同,前者是pytest_runtest_setup,而fixture用的pytest hook是pytest_fixture_setup)。

需要注意的是’xunit-style setup’针对的scope有module, class和method, 是没有session这个scope的。假如你有一个需求是所有tests执行前进行setup,所有tests执行完后teardown,则’xunit-style setup’无法满足这种需求(即使你通过设置基类中的setup_classteardown_class方法,再通过测试类来继承这个类,实际是会在每个测试类执行前都执行一次setup_class的(teardown_class也一样))。要实现这种需求只有以下两种方式:

  • 使用pytest fixture。在测试类的基类上放一个scope=session的fixture(使用@pytest.mark.usefixtures来放)。
  • 使用pytest hooks。写一个pytest plugin来实现比如pytest_runtestloop这样的hook,在其中进行setup的工作(参考pytest hook spec)。

Relation between pytest fixture and pytest hook

首先,pytest fixture本身就是通过hook来实现的。fixture的setup是通过pytest_fixture_setup这个hook来实现的,而teardown则是通过pytest_runtest_teardown来实现的。

再者,从fixture的定义来看,也和hook方法非常像,只是它们的注册方式不一样:即pytest定义了一套hook方法的规则(方法名要以pytest_开头等),fixture又自己定义了一套规则(要有@pytest.fixture装饰器),两套规则可以完全兼容,并形成了现在pytest的样子。

Conclusion

从用户角度来看,根本不需要关心’xunit-style setup’是怎么实现的,只要知道这种方式好用就行。你可以把它理解为一种特殊的fixture(不需要fixture装饰器,scope天然绑定且setup和teardown分开定义了)或者是一种特殊的hook方法( 只不过没有以pytest_开头,也不会被注册为pytest plugin)。总之,在design你的测试框架时知道有这样一个好用的东西就行。

Play Python Library之pytest--plugin篇

| Comment

Register as pytest plugin

通过pytest --trace-config命令可以查看当前pytest中所有的plugin。

比如:

➜  demo pytest --trace-config
PLUGIN registered: <_pytest.config.PytestPluginManager object at 0x10cd27a90>
PLUGIN registered: <_pytest.config.Config object at 0x10cfc20d0>
PLUGIN registered: <module '_pytest.mark' from '/usr/local/lib/python2.7/site-packages/_pytest/mark.pyc'>
PLUGIN registered: <module '_pytest.main' from '/usr/local/lib/python2.7/site-packages/_pytest/main.pyc'>
PLUGIN registered: <module '_pytest.terminal' from '/usr/local/lib/python2.7/site-packages/_pytest/terminal.pyc'>
PLUGIN registered: <module '_pytest.runner' from '/usr/local/lib/python2.7/site-packages/_pytest/runner.pyc'>
PLUGIN registered: <module '_pytest.python' from '/usr/local/lib/python2.7/site-packages/_pytest/python.pyc'>
PLUGIN registered: <module '_pytest.fixtures' from '/usr/local/lib/python2.7/site-packages/_pytest/fixtures.pyc'>
PLUGIN registered: <module '_pytest.debugging' from '/usr/local/lib/python2.7/site-packages/_pytest/debugging.pyc'>
PLUGIN registered: <module '_pytest.unittest' from '/usr/local/lib/python2.7/site-packages/_pytest/unittest.pyc'>
PLUGIN registered: <module '_pytest.capture' from '/usr/local/lib/python2.7/site-packages/_pytest/capture.pyc'>
PLUGIN registered: <module '_pytest.skipping' from '/usr/local/lib/python2.7/site-packages/_pytest/skipping.pyc'>
PLUGIN registered: <module '_pytest.tmpdir' from '/usr/local/lib/python2.7/site-packages/_pytest/tmpdir.pyc'>
PLUGIN registered: <module '_pytest.monkeypatch' from '/usr/local/lib/python2.7/site-packages/_pytest/monkeypatch.pyc'>
PLUGIN registered: <module '_pytest.recwarn' from '/usr/local/lib/python2.7/site-packages/_pytest/recwarn.pyc'>
PLUGIN registered: <module '_pytest.pastebin' from '/usr/local/lib/python2.7/site-packages/_pytest/pastebin.pyc'>
PLUGIN registered: <module '_pytest.helpconfig' from '/usr/local/lib/python2.7/site-packages/_pytest/helpconfig.pyc'>
PLUGIN registered: <module '_pytest.nose' from '/usr/local/lib/python2.7/site-packages/_pytest/nose.pyc'>
PLUGIN registered: <module '_pytest.assertion' from '/usr/local/lib/python2.7/site-packages/_pytest/assertion/__init__.pyc'>
PLUGIN registered: <module '_pytest.junitxml' from '/usr/local/lib/python2.7/site-packages/_pytest/junitxml.pyc'>
PLUGIN registered: <module '_pytest.resultlog' from '/usr/local/lib/python2.7/site-packages/_pytest/resultlog.pyc'>
PLUGIN registered: <module '_pytest.doctest' from '/usr/local/lib/python2.7/site-packages/_pytest/doctest.pyc'>
PLUGIN registered: <module '_pytest.cacheprovider' from '/usr/local/lib/python2.7/site-packages/_pytest/cacheprovider.pyc'>
PLUGIN registered: <module '_pytest.freeze_support' from '/usr/local/lib/python2.7/site-packages/_pytest/freeze_support.pyc'>
PLUGIN registered: <module '_pytest.setuponly' from '/usr/local/lib/python2.7/site-packages/_pytest/setuponly.pyc'>
PLUGIN registered: <module '_pytest.setupplan' from '/usr/local/lib/python2.7/site-packages/_pytest/setupplan.pyc'>
PLUGIN registered: <module 'pytest_pep8' from '/usr/local/lib/python2.7/site-packages/pytest_pep8.py'>
PLUGIN registered: <_pytest.capture.CaptureManager instance at 0x10debc5f0>
PLUGIN registered: <Session 'demo'>
PLUGIN registered: <_pytest.cacheprovider.LFPlugin instance at 0x10decb680>
PLUGIN registered: <_pytest.terminal.TerminalReporter instance at 0x10debc368>
PLUGIN registered: <_pytest.fixtures.FixtureManager instance at 0x10decbd88>

以上,似乎看的有点晕,但其实大部分都是pytest自带的plugin(通过它们的文件路径也可以大概看出来,除了pytest_pep8其他都是_pytest文件夹下的)。在pytest中,所谓plugin其实就是能被pytest发现的一些带有pytest hook方法的文件或对象。

其实官方文档也提到了pytest plugin加载的几种方式:

pytest loads plugin modules at tool startup in the following way:

  • by loading all builtin plugins

  • by loading all plugins registered through setuptools entry points.

  • by pre-scanning the command line for the -p name option and loading the specified plugin before actual command line parsing.

  • by loading all conftest.py files as inferred by the command line invocation:

    • if no test paths are specified use current dir as a test path
    • if exists, load conftest.py and test*/conftest.py relative to the directory part of the first test path.

    Note that pytest does not find conftest.py files in deeper nested sub directories at tool startup. It is usually a good idea to keep your conftest.py file in the top level test or project root directory.

  • by recursively loading all plugins specified by the pytest_plugins variable in conftest.py files

以下,是更详细的一些说明:

  • 通过entry points,也就是我们通常pip install的一些pytest plugin注册到pytest的方式。

    这是通过PluginManager.load_setuptools_entrypoints方法来加载的,通过断点可以进入这个方法查看所有由此加载的plugin(这里的entrypoint_name可以看到就是’pytest11’):

    for ep in iter_entry_points(entrypoint_name):
        print ep
    
  • 通过conftest.py的方式:这种方式其实就是在conftest.py中添加pytest的hook方法,把conftest.py本身作为plugin。

  • 通过设置pytest_plugins变量的方式:这种方法最为tricky,比如说在conftest.py中添加下面的这一行代码就把pytest_platform_test(当然这个文件本身要求能在当前路径被import)这个plugin给注册到pytest里了。

    pytest_plugins = ['pytest_platform_test']
    
Read more

Ansible实践心得

| Comment

以下针对Ansible 2.1.1.0,对其他版本的Ansible可能不适用。

  1. Ansible是的执行原理是将python代码传输到远端,然后在远端执行。因此:

    • 如果你要写Ansible的module,想要对远端的某个文件写入,只需要在module的代码里:

      with open(file_path, 'w') as f:
           f.write(content)
      

      像上述这样就可以了。

    • 如果你写的module需要使用一些外部的资源,要注意这些外部资源Ansible是不会自动帮你传输到远端的。比如我曾经想调用一个python文件:

      remote_hostname | FAILED! => {
          "changed": false, 
          "failed": true, 
          "module_stderr": "", 
          "module_stdout": "File \"/tmp/ansible_C9llBm/ansible_module_mymodule.py\", line 10, in <module>\r\n    import default\r\nImportError: No module named default\r\n", 
          "msg": "MODULE FAILURE", 
          "parsed": false
      }
      
  2. Ansible现在不支持从一个module中调用其他module。所以如果是用到多个module的组合的话,还是写在playbook里吧。

    If you are asking ‘how can I have a module execute other modules’ … you want to write a role.

  3. Ansible是可以对每个host使用不同的password等设置的。例子如下:

    [real_group]
    remote_host1 ansible_connection=ssh ansible_ssh_user=root1 ansible_ssh_pass=password1
    remote_host2 ansible_connection=ssh ansible_ssh_user=root2 ansible_ssh_pass=password2
    

    如果你觉得上述方式不直观,也可以把每个机器单独设为一个group,再把所有机器用一个group包起来,就像这样:

    [remote_host1]
    remote_hostname1
    
    [remote_host1:vars]
    ansible_connection=ssh
    ansible_ssh_user=root
    ansible_ssh_pass=password1
    
    [remote_host2]
    remote_hostname2
    
    [remote_host2:vars]
    ansible_connection=ssh
    ansible_ssh_user=root
    ansible_ssh_pass=password2
    
    [real_group:children]
    remote_host1
    remote_host2
    
  4. Ansible的module只能获取传递进去的参数,而没法知道hosts里面定义的内容,即如果想要module根据host的信息来搞事情,也必须要通过参数传递进去(这个design的module开发体验很差!)。

  5. 想要在python代码里执行Ansible的playbook,可参考How to use Ansible 2.0 Python API to run a Playbook?

Read more

Play Python Library之pytest--mark篇

| Comment

Introduction

pytest.mark主要是用来对test方法进行标记用的一个装饰器。标记的作用就是在使用pytest跑测试代码的时候可以选择性地执行部分test方法。

比如test代码如下:

import pytest


@pytest.mark.old_test
def test_one():
    assert False


@pytest.mark.new_test
def test_two():
    assert False


@pytest.mark.not_run
def test_three():
    assert False

通过使用-m参数可以让pytest选择性的执行部分test方法:

➜  python-demo git:(master) ✗ pytest demo_pytest/test_mark_0.py -m "not not_run"
==================================== test session starts ====================================
platform darwin -- Python 2.7.11, pytest-3.0.2, py-1.4.31, pluggy-0.3.1
rootdir: /Users/CYu/Code/Python/python-demo/demo_pytest, inifile: pytest.ini
plugins: pep8-1.0.6, splunk-rack-1.1.6
collected 3 items

demo_pytest/test_mark_0.py FF

========================================= FAILURES ==========================================
_________________________________________ test_one __________________________________________

    @pytest.mark.old_test
    def test_one():
>       assert False
E       assert False

demo_pytest/test_mark_0.py:13: AssertionError
_________________________________________ test_two __________________________________________

    @pytest.mark.new_test
    def test_two():
>       assert False
E       assert False

demo_pytest/test_mark_0.py:18: AssertionError
==================================== 1 tests deselected =====================================
========================== 2 failed, 1 deselected in 0.03 seconds ===========================
Read more

Play Python Library之pytest--fixture篇

| Comment

Introduction

pytest中,一个fixture其实就是一个函数,函数名就是fixture的名称。关于fixture,pytest在run一个test方法的时候,大概流程如下:

  1. 收集该test方法的作用域内的所有fixture; 这个作用域包括该test case所属的class,module等不同作用域的叠加(遇到同名的fixture,更里层的会覆盖更外层的):

    The discovery of fixtures functions starts at test classes, then test modules, then conftest.py files and finally builtin and third party plugins.

  2. 收集该test方法的pytest参数(通过pytest.mark.parametrize来定义);

  3. 对该test方法的输入根据该作用域内的fixture名称或pytest参数(遇到pytest参数和fixture同名的,pytest参数会覆盖fixture)进行填充后执行。

pytest中的fixture提供了一个很好的对象管理方式,我们可以将测试代码中经常用到的一些对象定义为fixture来统一进行管理,而省去了很多重复的代码(这点就像with-statement,不但减少了代码量,而且避免了代码中类似资源未正确释放的情况)。

从我的理解而言,fixture会适用于以下几类对象:

  • 资源类的对象。比如网络资源,fixture中可以进行网络的连接、断开等操作。
  • 全局变量。比如一些test共享的信息,当然通过类的继承也可以达到共享的作用。

关于fixture的样子,这里沿用pytest官方的例子如下:

import smtplib
import pytest

@pytest.fixture(scope="module")
def smtp(request):
    smtp = smtplib.SMTP("smtp.gmail.com")
    yield smtp  # provide the fixture value
    print("teardown smtp")
    smtp.close()

其中,在fixture定义中yield之后的语句是会在fixture对象超出定义的scope时执行的。

以上,可以看出一个fixture的几个特点:

  • 必须是一个有pytest.fixture的装饰器的函数;
  • 该函数一般会return或yield一个返回值;
  • 该函数在yield/return语句之前的操作都可以看做是对要返回对象的tear up的过程,而在yield(这里return不行)语句之后的操作都可以看做是对返回对象的tear down的过程;
  • 该函数可以有一个输入参数,这个输入参数包含一些test方法相关的信息。
Read more
| Page 19 of 25 |