Merge branch 'AT5-370' into 'master'

Request to merge AT5-370 branch with master

See merge request ska-telescope/csp-lmc-subelement!5

.gitlab-ci.yml

@@ -56,12 +56,11 @@ list_dependencies:
       - public
 
 linting:
-  image: nexus.engageska-portugal.pt/ska-docker/ska-python-buildenv:latest
-  tags:
-    - docker-executor
   stage: linting
   script:
-    - make lint
+    - pip3 install pylint2junit
+    - pylint --output-format=parseable cspse | tee ./build/code_analysis.stdout
+    - pylint --output-format=pylint2junit.JunitReporter cspse > ./build/reports/linting.xml;
   when: always
   artifacts:
     paths:

.pylintrc

@@ -28,7 +28,7 @@ limit-inference-results=100
 
 # List of plugins (as comma separated values of python modules names) to load,
 # usually to register additional checkers.
-load-plugins=pylint_junit
+load-plugins=
 
 # Pickle collected data for later comparisons.
 persistent=yes
@@ -95,7 +95,7 @@ evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / stateme
 # Set the output format. Available formats are text, parseable, colorized, json
 # and msvs (visual studio). You can also give a reporter class, e.g.
 # mypackage.mymodule.MyReporterClass.
-output-format=junit
+output-format=text
 
 # Tells whether to display a full report or only the messages.
 reports=yes

README.md

-SKA Python Skeleton Project
-===========================
+SKA CSP-LMC-SUBELEMENT Abstract Class
+=====================================
 
 [![Documentation Status](https://readthedocs.org/projects/ska-python-skeleton/badge/?version=latest)](https://developer.skatelescope.org/projects/skeleton/en/latest/?badge=latest)
 
-Briefly describe your project here
+## Table of contents
+* [Description](#description)
+* [Getting started](#getting-started)
+* [Repository](#repository)
+* [Prerequisities](#prerequisities)
+* [Running tests](#running-tests)
+* [Known bugs](#known-bugs)
+* [Troubleshooting](#troubleshooting)
+* [License](#license)
 
-Requirements
-------------
+## Description
 
-The system used for development needs to have Python 3 and `pip` installed.
+This project contains the `CSP.LMC.SUBELEMENT` prototype. It includes a
+single class:
 
-Install 
--------
+* the `CspSubElementMaster` device: based on the `CspMaster` class. The 
+`CspSubElementMaster` represents a primary point of contact for CSP
+SubElement Monitor and Control.  It implements CSP SubElement state and
+mode indicators and a limited set of housekeeping commands.
+It is intended to connect to the various subcomponent of SubElement. This
+can be accomplished directly or by means of a _caching_ device, called
+`Rack`. Of course it is a device collector and can or cannot correspond to a
+physical rack.
 
-**Always** use a virtual environment. [Pipenv](https://pipenv.readthedocs.io/en/latest/) is now Python's officially
-recommended method, but we are not using it for installing requirements when building on the CI Pipeline. You are encouraged to use your preferred environment isolation (i.e. `pip`, `conda` or `pipenv` while developing locally.
 
-For working with `Pipenv`, follow these steps at the project root:
+## Getting started
 
-First, ensure that `~/.local/bin` is in your `PATH` with:
-```bash
-> echo $PATH
-```
+The project can be found in the SKA gitlab repository.
+
+To get a local copy of the project:
 
-In case `~/.local/bin` is not part of your `PATH` variable, under Linux add it with:
 ```bash
-> export PATH=~/.local/bin:$PATH
+git clone https://gitlab.com/ska-telescope/csp-lmc-subelement.git
 ```
-or the equivalent in your particular OS.
+## Prerequisities
 
-Then proceed to install pipenv and the required environment packages:
+* A TANGO development environment properly configured, as described in [SKA developer portal](https://developer.skatelescope.org/en/latest/tools/tango-devenv-setup.html)
 
-```bash
-> pip install pipenv # if you don't have pipenv already installed on your system
-> pipenv install
-> pipenv shell
-```
+* [SKA Base classes](https://gitlab.com/ska-telescope/lmc-base-classes)
 
-You will now be inside a pipenv shell with your virtual environment ready.
 
-Use `exit` to exit the pipenv environment.
+## Repository organization
 
+The `CSP.LMC.SUBELEMENT` repository is organized in a single code tree. The
+hierarchy contains:
 
-Testing
--------
+* _cspse_: contains the specific project TANGO Device Class files
+* _pogo_: contains the POGO files of the TANGO Device Classes of the project
+* _docker_: contains the `docker`, `docker-compose` and `dsconfig` configuration files as well as the Makefile to generate the docker image and run the tests.
+* _tests_: contains the test
 
-* Put tests into the `tests` folder
-* Use [PyTest](https://pytest.org) as the testing framework
-  - Reference: [PyTest introduction](http://pythontesting.net/framework/pytest/pytest-introduction/)
-* Run tests with `python setup.py test`
-  - Configure PyTest in `setup.py` and `setup.cfg`
-* Running the test creates the `htmlcov` folder
-    - Inside this folder a rundown of the issues found will be accessible using the `index.html` file
-* All the tests should pass before merging the code 
+## Running tests
 
- Code analysis
- -------------
- * Use [Pylint](https://www.pylint.org) as the code analysis framework
- * By default it uses the [PEP8 style guide](https://www.python.org/dev/peps/pep-0008/)
- * Use the provided `code-analysis.sh` script in order to run the code analysis in the `module` and `tests`
- * Code analysis should be run by calling `pylint ska_python_skeleton`. All pertaining options reside under the `.pylintrc` file.
- * Code analysis should only raise document related warnings (i.e. `#FIXME` comments) before merging the code
+To run the internal test go to `tests` directory and execute:
 
-Writing documentation
- --------------------
- * The documentation generator for this project is derived from SKA's [SKA Developer Portal repository](https://github.com/ska-telescope/developer.skatelescope.org)
- * The documentation can be edited under `./docs/src`
- * If you want to include only your README.md file, create a symbolic link inside the `./docs/src` directory if the existing one does not work:
- ```bash
-$ cd docs/src
-$ ln -s ../../README.md README.md
-```
- * In order to build the documentation for this specific project, execute the following under `./docs`:
 ```bash
-$ make html
+make test
 ```
-* The documentation can then be consulted by opening the file `./docs/build/html/index.html`
+
+## Known bugs
+
+_Still none_
+
+## Troubleshooting
+
+TBD
+
+## License
+See the LICENSE file for details.
+
+

cspse/lmc/__init__.py

+__all__ = (
+    "CspSubElementMaster"
+)
+
+from .subelement_master import CspSubElementMaster
+

cspse/lmc/decorators.py

+import functools
+import tango
+
+tasks = {}
+
+# note: f.__name__ is of type is_XXX_allowed
+# f.__name__[3:-8] select the command name
+# this decorator build a dictionary with the command name as key and
+# the handler as value.
+task = lambda f:tasks.setdefault(f.__name__[3:-8], f)
+
+@task
+def is_standby_allowed(device_instance):
+    """
+    Allowed method for Standby method.
+    Command *Standby* is allowed when the device *State* is ON or
+    STANDBY.
+
+    :return: True if the method is allowed, otherwise False.
+    """
+    if device_instance.get_state() in [tango.DevState.ON, 
+                                        tango.DevState.STANDBY]:
+        return True
+    return False
+
+@task
+def is_on_allowed(device_instance):
+    """
+    Allowed method for On method.
+    Command *On* is allowed when the device *State* is ON or
+    STANDBY.
+
+    :return: True if the method is allowed, otherwise False.
+    """
+    if device_instance.get_state() in [tango.DevState.ON, 
+                                        tango.DevState.STANDBY]:
+        return True
+    return False
+
+@task
+def is_off_allowed(device_instance):
+    """
+    Allowed method for Off method.
+    Command *Off* is allowed when the device *State* is OFF or
+    STANDBY.
+
+    :return: True if the method is allowed, otherwise False.
+    """
+    if device_instance.get_state() in [tango.DevState.OFF, 
+                                       tango.DevState.STANDBY]:
+        return True
+    return False
+
+def is_command_allowed(device_instance, cmd_name):
+    """
+    Call the allowed method for the command name specified
+    as input argument
+    :param device_istance: the TANGO device instance
+    :param cmd_name: the name of command to execute
+
+    :return: True/False
+    """
+    tasks[cmd_name](device_instance)
+
+class IsCommandAllowed(object):
+    """
+    Class designed to be a decorator for the Master power methods.
+    The *decorator function* performs a check on the input argument
+    to control if the command is issued on the whole sub-element.
+    If this is the case, it checks the State of the sub-element Master
+    device and rejects the command accordingly to the State
+    machine setting.
+
+    :raise: tango.DevFailed exception if the command can't be executed 
+    """
+    def __init__(self, *args, **kwargs):
+        # store the decorators parameters:
+        # args: the list of sub-element attributes to subscribe, to track the
+        #       sub-element command progress and detect a command timeout
+        # kwargs: a dictionary: key ="cmd_name",
+        #                       value = command to execute ('on', 'off'...)
+        self._args = args
+        self._kwargs = kwargs
+
+    def __call__(self, f):
+        @functools.wraps(f)
+        def input_args_check(*args, **kwargs):
+            # the Master device instance
+            dev_instance = args[0]
+            # the command name
+            cmd_to_exec = f.__name__
+            # the command input argument
+            input_arg = args[1]
+            #device_list = input_arg
+            # Note: device list is a reference to args[1]: changing
+            # device_list content, args[1] changes accordingly!
+            num_of_devices = len(input_arg)
+            if num_of_devices == 0:
+                # check the device State: if it not the proper value the command is
+                # not executed
+                if not is_command_allowed(dev_instance, cmd_to_exec.lower()):
+                    msg = "Command {} can't be executed when the device is {}".format(cmd_to_exec,
+                                                                                       dev_instance.get_state())
+                    tango.Except.throw_exception("Command failure",msg,
+                                             "IsCommandAllowed decorator",
+                                             tango.ErrSeverity.ERR)
+            return f(*args, **kwargs)
+        return input_args_check

cspse/lmc/release.py

+# -*- coding: utf-8 -*-
+#
+# This file is part of the CspSubElementMaster project
+#
+#
+#
+# Distributed under the terms of the BSD-3-Clause license.
+# See LICENSE.txt for more info.
+
+"""Release information for Python Package"""
+
+name = """csp-lmc-subelement"""
+version = "0.1.0"
+version_info = version.split(".")
+description = """SKA CSP Sub-element LMC"""
+author = "INAF-OAA"
+author_email = "elisabetta.giani@inaf.it"
+license = """BSD-3-Clause"""
+url = """https://gitlab.com/ska-telescope/csp-lmc.git"""
+copyright = """INAF, SKA Telescope"""

docker-requirements.txt

+--extra-index-url https://nexus.engageska-portugal.pt/repository/pypi/simple
 docutils
 markupsafe
 pygments
@@ -14,3 +15,4 @@ sphinx-rtd-theme
 sphinxcontrib-websupport
 pipdeptree
 pylint_junit
+csp-lmc-common >= 0.5.3

docker/.make/.make-release-support

+#!/bin/bash
+#
+#   Copyright 2015  Xebia Nederland B.V.
+#   Modifications copyright 2019 SKA Organisation
+#
+#   Licensed under the Apache License, Version 2.0 (the "License");
+#   you may not use this file except in compliance with the License.
+#   You may obtain a copy of the License at
+#
+#       http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS,
+#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#   See the License for the specific language governing permissions and
+#   limitations under the License.
+#
+function hasChanges() {
+	test -n "$(git status -s .)"
+}
+
+function getRelease() {
+	awk -F= '/^release=/{print $2}' .release
+}
+
+function getBaseTag() {
+		sed -n -e "s/^tag=\(.*\)$(getRelease)\$/\1/p" .release
+}
+
+function getTag() {
+	if [ -z "$1" ] ; then
+		awk -F= '/^tag/{print $2}' .release
+	else
+		echo "$(getBaseTag)$1"
+	fi
+}
+
+function setRelease() {
+	if [ -n "$1" ] ; then
+		sed -i.x -e "s/^tag=.*/tag=$(getTag $1)/" .release
+		sed -i.x -e "s/^release=.*/release=$1/g" .release
+		rm -f .release.x
+		runPreTagCommand "$1"
+	else
+		echo "ERROR: missing release version parameter " >&2
+		return 1
+	fi
+}
+
+function runPreTagCommand() {
+	if [ -n "$1" ] ; then
+		COMMAND=$(sed -n -e "s/@@RELEASE@@/$1/g" -e 's/^pre_tag_command=\(.*\)/\1/p' .release)
+		if [ -n "$COMMAND" ] ; then
+			if ! OUTPUT=$(bash -c "$COMMAND" 2>&1) ; then echo $OUTPUT >&2 && exit 1 ; fi
+		fi
+	else
+		echo "ERROR: missing release version parameter " >&2
+		return 1
+	fi
+}
+
+function tagExists() {
+	tag=${1:-$(getTag)}
+	test -n "$tag" && test -n "$(git tag | grep "^$tag\$")"
+}
+
+function differsFromRelease() {
+	tag=$(getTag)
+	! tagExists $tag || test -n "$(git diff --shortstat -r $tag .)"
+}
+
+function getVersion() {
+	result=$(getRelease)
+
+	if differsFromRelease; then
+		result="$result-$(git log -n 1 --format=%h .)"
+	fi
+
+	if hasChanges ; then
+		result="$result-dirty"
+	fi
+	echo $result
+}
+
+function nextPatchLevel() {
+	version=${1:-$(getRelease)}
+	major_and_minor=$(echo $version | cut -d. -f1,2)
+	patch=$(echo $version | cut -d. -f3)
+	version=$(printf "%s.%d" $major_and_minor $(($patch + 1)))
+	echo $version
+}
+
+function nextMinorLevel() {
+	version=${1:-$(getRelease)}
+	major=$(echo $version | cut -d. -f1);
+	minor=$(echo $version | cut -d. -f2);
+	version=$(printf "%d.%d.0" $major $(($minor + 1))) ;
+	echo $version
+}
+
+function nextMajorLevel() {
+	version=${1:-$(getRelease)}
+	major=$(echo $version | cut -d. -f1);
+	version=$(printf "%d.0.0" $(($major + 1)))
+	echo $version
+}

docker/.make/Makefile.mk

+#
+#   Copyright 2015  Xebia Nederland B.V.
+#
+#   Licensed under the Apache License, Version 2.0 (the "License");
+#   you may not use this file except in compliance with the License.
+#   You may obtain a copy of the License at
+#
+#       http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS,
+#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#   See the License for the specific language governing permissions and
+#   limitations under the License.
+#
+
+ifeq ($(strip $(PROJECT)),)
+  NAME=$(shell basename $(CURDIR))
+else
+  NAME=$(PROJECT)
+endif
+
+RELEASE_SUPPORT := $(shell dirname $(abspath $(lastword $(MAKEFILE_LIST))))/.make-release-support
+
+ifeq ($(strip $(DOCKER_REGISTRY_HOST)),)
+  DOCKER_REGISTRY_HOST = nexus.engageska-portugal.pt
+endif
+
+ifeq ($(strip $(DOCKER_REGISTRY_USER)),)
+  DOCKER_REGISTRY_USER = ska-docker
+endif
+
+IMAGE=$(DOCKER_REGISTRY_HOST)/$(DOCKER_REGISTRY_USER)/$(NAME)
+
+#VERSION = release version + git sha
+VERSION=$(shell . $(RELEASE_SUPPORT) ; getVersion)
+
+#BASE_VERSION
+BASE_VERSION=$(shell . $(RELEASE_SUPPORT) ; getRelease)
+
+#TAG = project name + release version
+TAG=$(shell . $(RELEASE_SUPPORT); getTag)
+
+#DEFAULT_TAG = image name + BASE_VERSION
+DEFAULT_TAG=$(IMAGE):$(BASE_VERSION)
+
+
+SHELL=/bin/bash
+
+DOCKER_BUILD_CONTEXT=..
+DOCKER_FILE_PATH=Dockerfile
+
+.PHONY: pre-build docker-build post-build build release patch-release minor-release major-release tag check-status check-release showver \
+	push pre-push do-push post-push
+
+build: pre-build docker-build post-build  ## build the application image
+
+pre-build:
+
+post-build:
+
+pre-push:
+
+post-push:
+
+docker-build: .release
+	@echo "Building image: $(IMAGE):$(VERSION)"
+	@echo "NAME: $(NAME)"
+	docker build $(DOCKER_BUILD_ARGS) -t $(IMAGE):$(VERSION) $(DOCKER_BUILD_CONTEXT) -f $(DOCKER_FILE_PATH) --build-arg DOCKER_REGISTRY_HOST=$(DOCKER_REGISTRY_HOST) --build-arg DOCKER_REGISTRY_USER=$(DOCKER_REGISTRY_USER)
+	@DOCKER_MAJOR=$(shell docker -v | sed -e 's/.*version //' -e 's/,.*//' | cut -d\. -f1) ; \
+	DOCKER_MINOR=$(shell docker -v | sed -e 's/.*version //' -e 's/,.*//' | cut -d\. -f2) ; \
+	if [ $$DOCKER_MAJOR -eq 1 ] && [ $$DOCKER_MINOR -lt 10 ] ; then \
+		echo docker tag -f $(IMAGE):$(VERSION) $(IMAGE):latest ;\
+		docker tag -f $(IMAGE):$(VERSION) $(IMAGE):latest ;\
+	else \
+		echo docker tag $(IMAGE):$(VERSION) $(IMAGE):latest ;\
+		docker tag $(IMAGE):$(VERSION) $(IMAGE):latest ; \
+	fi
+
+release: check-status check-release build push
+
+push: pre-push do-push post-push  ## push the image to the Docker registry
+
+do-push: ## Push the image tagged as $(IMAGE):$(VERSION) and $(DEFAULT_TAG)
+	@echo -e "Tagging: $(IMAGE):$(VERSION) -> $(DEFAULT_TAG)"
+	docker tag $(IMAGE):$(VERSION) $(DEFAULT_TAG)
+	@echo -e "Pushing: $(IMAGE):$(VERSION)"
+	docker push $(IMAGE):$(VERSION)
+	@echo -e "Pushing: $(DEFAULT_TAG)"
+	docker push $(DEFAULT_TAG)
+
+tag_latest: do-push ## Tag the images as latest
+	@echo "Tagging: $(DEFAULT_TAG) -> $(IMAGE):latest"
+	@docker tag $(DEFAULT_TAG) $(IMAGE):latest
+
+push_latest: tag_latest ## Push the image tagged as :latest
+	@echo "Pushing: $(IMAGE):latest"
+	@docker push $(IMAGE):latest
+
+snapshot: build push
+
+showver: .release
+	@. $(RELEASE_SUPPORT); getVersion
+
+bump-patch-release: VERSION := $(shell . $(RELEASE_SUPPORT); nextPatchLevel)
+bump-patch-release: .release tag
+
+bump-minor-release: VERSION := $(shell . $(RELEASE_SUPPORT); nextMinorLevel)
+bump-minor-release: .release tag
+
+bump-major-release: VERSION := $(shell . $(RELEASE_SUPPORT); nextMajorLevel)
+bump-major-release: .release tag
+
+patch-release: tag-patch-release release
+	@echo $(VERSION)
+
+minor-release: tag-minor-release release
+	@echo $(VERSION)
+
+major-release: tag-major-release release
+	@echo $(VERSION)
+
+tag: TAG=$(shell . $(RELEASE_SUPPORT); getTag $(VERSION))
+tag: check-status
+#	@. $(RELEASE_SUPPORT) ; ! tagExists $(TAG) || (echo "ERROR: tag $(TAG) for version $(VERSION) already tagged in git" >&2 && exit 1) ;
+	@. $(RELEASE_SUPPORT) ; setRelease $(VERSION)
+#	git add .
+#	git commit -m "bumped to version $(VERSION)" ;
+#	git tag $(TAG) ;
+#	@ if [ -n "$(shell git remote -v)" ] ; then git push --tags ; else echo 'no remote to push tags to' ; fi
+
+check-status:
+	@. $(RELEASE_SUPPORT) ; ! hasChanges || (echo "ERROR: there are still outstanding changes" >&2 && exit 1) ;
+
+check-release: .release
+	@. $(RELEASE_SUPPORT) ; tagExists $(TAG) || (echo "ERROR: version not yet tagged in git. make [minor,major,patch]-release." >&2 && exit 1) ;
+	@. $(RELEASE_SUPPORT) ; ! differsFromRelease $(TAG) || (echo "ERROR: current directory differs from tagged $(TAG). make [minor,major,patch]-release." ; exit 1)

docker/.release

+# -*- coding: utf-8 -*-
+#
+# This file is part of the CSP.LMC prototype project
+#
+#
+# Distributed under the terms of the BSD-3-Clause license.
+# See LICENSE.txt for more info.
+
+"""Release information for Python Package"""
+
+name = """csplmc-subelement"""
+version = "0.1.0"
+version_info = version.split(".")
+description = """SKA CSP.LMC Subelement Classes"""
+author = "E.G"
+author_email = "elisabetta.giani@inaf.it"
+license = """BSD-3-Clause"""
+url = """www.tango-controls.org"""
+copyright = """"""
+
+release=0.1.0
+tag=csp-lmc-subelement-0.1.0

docker/Dockerfile

+FROM nexus.engageska-portugal.pt/ska-docker/ska-python-buildenv:9.3.1 AS buildenv
+FROM nexus.engageska-portugal.pt/ska-docker/ska-python-runtime:9.3.1 AS runtime
+
+# create ipython profile to so that itango doesn't fail if ipython hasn't run yet
+RUN ipython profile create
+#RUN python3 -m pip install --user pytest-forked
+ENV PATH=/home/tango/.local/bin:$PATH
+#install csp-lmc-common with dependencies
+RUN python3 -m pip install -e . --user --extra-index-url https://nexus.engageska-portugal.pt/repository/pypi/simple
+
+CMD ["/venv/bin/python", "/app/cspse/lmc/SubElementMaster.py" ]

docker/Makefile

+#
+# Project makefile for a Tango project. You should normally only need to modify
+# DOCKER_REGISTRY_USER and PROJECT below.
+#
+
+#
+# DOCKER_REGISTRY_HOST, DOCKER_REGISTRY_USER and PROJECT are combined to define
+# the Docker tag for this project. The definition below inherits the standard
+# value for DOCKER_REGISTRY_HOST (=rnexus.engageska-portugal.pt) and overwrites
+# DOCKER_REGISTRY_USER and PROJECT to give a final Docker tag of
+# nexus.engageska-portugal.pt/tango-example/csplmc
+#
+
+DOCKER_REGISTRY_USER:=ska-docker
+PROJECT = csp-lmc-subelement
+
+#
+# include makefile to pick up the standard Make targets, e.g., 'make build'
+# build, 'make push' docker push procedure, etc. The other Make targets
+# ('make interactive', 'make test', etc.) are defined in this file.
+#
+include .make/Makefile.mk
+
+#
+# IMAGE_TO_TEST defines the tag of the Docker image to test
+#
+IMAGE_TO_TEST = $(DOCKER_REGISTRY_HOST)/$(DOCKER_REGISTRY_USER)/$(PROJECT):latest
+
+#
+# CACHE_VOLUME is the name of the Docker volume used to cache eggs and wheels
+# used during the test procedure. The volume is not used during the build
+# procedure
+#
+CACHE_VOLUME = $(PROJECT)-test-cache
+
+# optional docker run-time arguments
+DOCKER_RUN_ARGS =
+
+#
+# Never use the network=host mode when running CI jobs, and add extra
+# distinguishing identifiers to the network name and container names to
+# prevent collisions with jobs from the same project running at the same
+# time.
+#
+ifneq ($(CI_JOB_ID),)
+NETWORK_MODE := tangonet-$(CI_JOB_ID)
+CONTAINER_NAME_PREFIX := $(PROJECT)-$(CI_JOB_ID)-
+else
+CONTAINER_NAME_PREFIX := $(PROJECT)-
+endif
+
+COMPOSE_FILES := $(wildcard *.yml)
+COMPOSE_FILE_ARGS := $(foreach yml,$(COMPOSE_FILES),-f $(yml))
+
+ifeq ($(OS),Windows_NT)
+    $(error Sorry, Windows is not supported yet)
+else
+	UNAME_S := $(shell uname -s)
+	ifeq ($(UNAME_S),Linux)
+		DISPLAY ?= :0.0
+		NETWORK_MODE ?= host
+		XAUTHORITY_MOUNT := /tmp/.X11-unix:/tmp/.X11-unix
+		XAUTHORITY ?= /hosthome/.Xauthority
+		# /bin/sh (=dash) does not evaluate 'docker network' conditionals correctly
+		SHELL := /bin/bash
+	endif
+	ifeq ($(UNAME_S),Darwin)
+		IF_INTERFACE := $(shell netstat -nr | awk '{ if ($$1 ~/default/) { print $$6} }')
+		DISPLAY := $(shell ifconfig $(IF_INTERFACE) | awk '{ if ($$1 ~/inet$$/) { print $$2} }'):0
+		# network_mode = host doesn't work on MacOS, so fix to the internal network
+		NETWORK_MODE := tangonet
+		XAUTHORITY_MOUNT := $(HOME):/hosthome:ro
+		XAUTHORITY := /hosthome/.Xauthority
+	endif
+endif
+
+#
+# When running in network=host mode, point devices at a port on the host
+# machine rather than at the container.
+#
+ifeq ($(NETWORK_MODE),host)
+TANGO_HOST := $(shell hostname):10000
+MYSQL_HOST := $(shell hostname):3306
+else
+# distinguish the bridge network from others by adding the project name
+NETWORK_MODE := $(NETWORK_MODE)-$(PROJECT)
+TANGO_HOST := $(CONTAINER_NAME_PREFIX)databaseds:10000
+MYSQL_HOST := $(CONTAINER_NAME_PREFIX)tangodb:3306
+endif
+
+
+DOCKER_COMPOSE_ARGS := DISPLAY=$(DISPLAY) XAUTHORITY=$(XAUTHORITY) TANGO_HOST=$(TANGO_HOST) \
+		NETWORK_MODE=$(NETWORK_MODE) XAUTHORITY_MOUNT=$(XAUTHORITY_MOUNT) MYSQL_HOST=$(MYSQL_HOST) \
+		DOCKER_REGISTRY_HOST=$(DOCKER_REGISTRY_HOST) DOCKER_REGISTRY_USER=$(DOCKER_REGISTRY_USER) \
+		CONTAINER_NAME_PREFIX=$(CONTAINER_NAME_PREFIX) COMPOSE_IGNORE_ORPHANS=true
+
+#
+# Defines a default make target so that help is printed if make is called
+# without a target
+#
+.DEFAULT_GOAL := help
+
+#
+# defines a function to copy the ./test-harness directory into the container
+# and then runs the requested make target in the container. The container is:
+#
+#   1. attached to the network of the docker-compose test system
+#   2. uses a persistent volume to cache Python eggs and wheels so that fewer
+#      downloads are required
+#   3. uses a transient volume as a working directory, in which untarred files
+#      and test output can be written in the container and subsequently copied
+#      to the host
+#
+make = tar -c test-harness/ | \
+	   docker run -i --rm --network=$(NETWORK_MODE) \
+	   -e TANGO_HOST=$(TANGO_HOST) \
+	   -v $(CACHE_VOLUME):/home/tango/.cache \
+	   -v /build -w /build -u tango $(DOCKER_RUN_ARGS) $(IMAGE_TO_TEST) \
+	   bash -c "sudo chown -R tango:tango /build && \
+	   tar x --strip-components 1 --warning=all && \
+	   make TANGO_HOST=$(TANGO_HOST) $1"
+
+test: DOCKER_RUN_ARGS = --volumes-from=$(BUILD)
+test: build up ## test the application
+	@echo "BUILD: $(BUILD)"
+	$(INIT_CACHE)
+	$(call make,test); \
+	  status=$$?; \
+	  rm -fr build; \
+          #docker-compose $(COMPOSE_FILE_ARGS) logs;
+	  docker cp $(BUILD):/build .; \
+	  docker rm -f -v $(BUILD); \
+	  $(MAKE) down; \
+	  exit $$status
+
+lint: DOCKER_RUN_ARGS = --volumes-from=$(BUILD) 
+lint: build up ## lint the application (static code analysis)
+	$(INIT_CACHE)
+	$(call make,lint); \
+	status=$$?; \
+	docker cp $(BUILD):/build .; \
+	$(MAKE) down; \
+	exit $$status	  
+
+pull:  ## download the application image
+	docker pull $(IMAGE_TO_TEST)
+
+up: build  ## start develop/test environment
+ifneq ($(NETWORK_MODE),host)
+	docker network inspect $(NETWORK_MODE) &> /dev/null || ([ $$? -ne 0 ] && docker network create $(NETWORK_MODE))
+endif
+	#$(DOCKER_COMPOSE_ARGS) docker-compose $(COMPOSE_FILE_ARGS) pull
+	#to pull only the mid-cbf-mcs image remove comment on row below.
+	#docker pull $(DOCKER_REGISTRY_HOST)/$(DOCKER_REGISTRY_USER)/mid-cbf-mcs:latest
+	$(DOCKER_COMPOSE_ARGS) docker-compose -f se-tangodb.yml up -d
+	# put a sleep to wait TANGO DB 
+	@sleep 10
+	$(DOCKER_COMPOSE_ARGS) docker-compose $(COMPOSE_FILE_ARGS) up -d 
+
+piplock: build  ## overwrite Pipfile.lock with the image version
+	docker run $(IMAGE_TO_TEST) cat /app/Pipfile.lock > $(CURDIR)/Pipfile.lock
+
+interactive: up
+interactive:  ## start an interactive session using the project image (caution: R/W mounts source directory to /app)
+	docker run --rm -it -p 3000:3000 --name=$(CONTAINER_NAME_PREFIX)dev -e TANGO_HOST=$(TANGO_HOST) --network=$(NETWORK_MODE) \
+	  -v $(CURDIR):/app $(IMAGE_TO_TEST) /bin/bash
+
+down:  ## stop develop/test environment and any interactive session
+	docker ps | grep $(CONTAINER_NAME_PREFIX)dev && docker stop $(PROJECT)-dev || true
+	$(DOCKER_COMPOSE_ARGS) docker-compose $(COMPOSE_FILE_ARGS) down
+ifneq ($(NETWORK_MODE),host)
+	docker network inspect $(NETWORK_MODE) &> /dev/null && ([ $$? -eq 0 ] && docker network rm $(NETWORK_MODE)) || true
+endif
+
+dsconfigdump: up ## dump the entire configuration to the file dsconfig.json
+	docker exec -it $(CONTAINER_NAME_PREFIX)dsconfigdump python -m dsconfig.dump
+	docker exec -it $(CONTAINER_NAME_PREFIX)dsconfigdump python -m dsconfig.dump > dsconfig.json
+
+dsconfigadd: up ## Add a configuration json file (environment variable DSCONFIG_JSON_FILE) to the database
+	-docker exec -it $(CONTAINER_NAME_PREFIX)dsconfigdump json2tango -u -w -a $(DSCONFIG_JSON_FILE)
+
+dsconfigcheck: up ## check a json file (environment variable DSCONFIG_JSON_FILE) according to the project lib-maxiv-dsconfig json schema
+	-docker exec -it $(CONTAINER_NAME_PREFIX)dsconfigdump json2tango -a $(DSCONFIG_JSON_FILE)
+
+help:  ## show this help.
+	@grep -hE '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
+
+.PHONY: all test up down help
+
+# Creates Docker volume for use as a cache, if it doesn't exist already
+INIT_CACHE = \
+	docker volume ls | grep $(CACHE_VOLUME) || \
+	docker create --name $(CACHE_VOLUME) -v $(CACHE_VOLUME):/cache $(IMAGE_TO_TEST)
+
+# http://cakoose.com/wiki/gnu_make_thunks
+BUILD_GEN = $(shell docker create -v /build $(IMAGE_TO_TEST))
+BUILD = $(eval BUILD := $(BUILD_GEN))$(BUILD)

docker/config/csplmc_dsconfig.json

+{
+    "servers": {
+        "CspSubElementMaster": {
+            "master": {
+                "CspSubElementMaster": {
+                    "subelement/sub_elt/master": {
+                        "attribute_properties": {
+                            "adminMode": {
+                                "abs_change": [
+                                    "-1", 
+                                    "1"
+                                ],
+				"__value": [
+				    "0"
+				]
+                            }, 
+                            "healthState": {
+                                "abs_change": [
+                                    "-1", 
+                                    "1"
+                                ]
+                            } 
+                        }, 
+                        "properties": {
+			    "Rack": [
+				"subelement/rack/01",
+				"subelement/rack/02",
+				"subelement/rack/03",
+				"subelement/rack/04"
+                            ]
+                        }
+                    }
+                }
+            }
+        }, 
+        "CspCapabilityMonitor": {
+            "searchbeams": {
+                "CspCapabilityMonitor": {
+                    "subelement/capability/pss_pipeline_monitor": {
+                        "attribute_properties": {
+                            "adminMode": {
+				"__value": [
+				    "0"
+				]
+                            } 
+                        }, 
+                        "properties": {
+			    "CapabilityDevices": [
+				"subelement/pipelines/0001",
+				"subelement/pipelines/0002",
+				"subelement/pipelines/0003"
+			    ]
+                        }
+                    }
+                }
+            },
+            "timingbeams": {
+                "CspCapabilityMonitor": {
+                    "subelement/capability/pstbeams_monitor": {
+                        "attribute_properties": {
+                            "adminMode": {
+				"__value": [
+				    "0"
+				]
+                            } 
+                        }, 
+                        "properties": {
+			    "CapabilityDevices": [
+				"subelement/pstbeam/01",
+				"subelement/pstbeam/02",
+				"subelement/pstbeam/03"
+			    ]
+                        }
+                    }
+                }
+            },
+            "vlbibeams": {
+                "CspCapabilityMonitor": {
+                    "subelement/capability/vlbi_beams_monitor": {
+                        "attribute_properties": {
+                            "adminMode": {
+				"__value": [
+				    "0"
+				]
+                            } 
+                        }, 
+                        "properties": {
+			    "CapabilityDevices": [
+				"subelement/vlbi_beam/01",
+				"subelement/vlbi_beam/02",
+				"subelement/vlbi_beam/03"
+			    ]
+                        }
+                    }
+                }
+            }
+	},
+        "CspSubElementSubarray": {
+            "subarray1": {
+                "CspSubElementSubarray": {
+                    "subelement/sub_elt/subarray_01": {
+                        "attribute_properties": {
+                            "adminMode": {
+                                "abs_change": [
+                                    "-1", 
+                                    "1"
+                                ],
+				"__value": [
+				    "0"
+				]
+                            }, 
+                            "healthState": {
+                                "abs_change": [
+                                    "-1", 
+                                    "1"
+                                ]
+                            }, 
+                            "obsState": {
+                                "abs_change": [
+                                    "-1", 
+                                    "1"
+                                ]
+                            } 
+                        }, 
+                        "properties": {
+                            "SubID": [
+                                "1"
+                            ] 
+                        }
+                    }
+                }
+            },
+            "subarray2": {
+                "SubElementSubarray": {
+                    "subelement/sub_elt/subarray_02": {
+                        "attribute_properties": {
+                            "adminMode": {
+                                "abs_change": [
+                                    "-1", 
+                                    "1"
+                                ],
+				"__value": [
+				    "0"
+				]
+                            }, 
+                            "healthState": {
+                                "abs_change": [
+                                    "-1", 
+                                    "1"
+                                ]
+                            }, 
+                            "obsState": {
+                                "abs_change": [
+                                    "-1", 
+                                    "1"
+                                ]
+                            } 
+                        }, 
+                        "properties": {
+                            "SubID": [
+                                "2"
+                            ] 
+                        }
+                    }
+                }
+            }
+        }, 
+        "DataBaseds": {
+            "2": {
+                "DataBase": {
+                    "sys/database/2": {}
+                }
+            }
+        }, 
+        "TangoAccessControl": {
+            "1": {
+                "TangoAccessControl": {
+                    "sys/access_control/1": {}
+                }
+            }
+        }, 
+        "TangoTest": {
+            "test": {
+                "TangoTest": {
+                    "sys/tg_test/1": {}
+                }
+            }
+        }
+    }
+}

docker/se-lmc.yml

+#
+# Docker compose file for TANGO database and database device server
+#
+# Defines:
+#   - tangodb: MariaDB database with TANGO schema
+#   - databaseds: TANGO database device server
+#   - rsyslog-csplmc: rsyslog service for logger
+#   - cspmaster: CspMaster device
+#
+# Requires:
+#   - None
+#
+version: '2.2'
+
+services:
+  se_dsconfig:
+    image: nexus.engageska-portugal.pt/ska-docker/tango-dsconfig:latest
+    network_mode: ${NETWORK_MODE}
+    container_name: ${CONTAINER_NAME_PREFIX}se_dsconfig
+    depends_on:
+      - databaseds
+    environment:
+      - TANGO_HOST=${TANGO_HOST}
+    command: >
+      sh -c "wait-for-it.sh ${TANGO_HOST} --timeout=60 --strict --
+             json2tango -w -a -u data/config/csplmc_dsconfig.json && sleep infinity"
+    volumes:
+      - .:/data
+
+  sesubarray01:
+    image: ${DOCKER_REGISTRY_HOST}/${DOCKER_REGISTRY_USER}/csp-lmc-subelement:latest
+    network_mode: ${NETWORK_MODE}
+    container_name: ${CONTAINER_NAME_PREFIX}sesubarray01
+    depends_on:
+      - se_dsconfig
+      - semaster
+    environment:
+      - TANGO_HOST=${TANGO_HOST}
+    command: >
+      sh -c "wait-for-it.sh ${TANGO_HOST} --timeout=30 --strict --
+             retry --max=5 -- tango_admin --ping-device subelement/sub_elt/master &&\
+             /venv/bin/python -m cspse.lmc.SubElementSubarray subarray1"
+  sesubarray02:
+    image: ${DOCKER_REGISTRY_HOST}/${DOCKER_REGISTRY_USER}/csp-lmc-subelement:latest
+    network_mode: ${NETWORK_MODE}
+    container_name: ${CONTAINER_NAME_PREFIX}sesubarray02
+    depends_on:
+      - se_dsconfig
+      - semaster
+    environment:
+      - TANGO_HOST=${TANGO_HOST}
+    command: >
+      sh -c "wait-for-it.sh ${TANGO_HOST} --timeout=30 --strict --
+             retry --max=5 -- tango_admin --ping-device subelement/sub_elt/master &&\
+             /venv/bin/python -m cspse.lmc.SubElementSubarray subarray2"
+
+  semaster:
+    image: ${DOCKER_REGISTRY_HOST}/${DOCKER_REGISTRY_USER}/csp-lmc-subelement:latest
+    network_mode: ${NETWORK_MODE}
+    container_name: ${CONTAINER_NAME_PREFIX}semaster
+    depends_on:
+      - se_dsconfig
+    environment:
+      - TANGO_HOST=${TANGO_HOST}
+    command: >
+      sh -c "wait-for-it.sh ${TANGO_HOST} --timeout=30 --strict -- CspSubElementMaster master"
+

docker/se-tangodb.yml

+#
+# Docker compose file for TANGO database and database device server
+#
+# Defines:
+#   - tangodb: MariaDB database with TANGO schema
+#   - databaseds: TANGO database device server
+#
+# Requires:
+#   - None
+#
+version: '2.2'
+volumes:
+  se-tangodb: {}
+
+services:
+  tangodb:
+    image: nexus.engageska-portugal.pt/ska-docker/tango-db:latest
+    network_mode: ${NETWORK_MODE}
+    container_name: ${CONTAINER_NAME_PREFIX}tangodb
+    environment:
+      - MYSQL_ROOT_PASSWORD=secret
+      - MYSQL_DATABASE=tango
+      - MYSQL_USER=tango
+      - MYSQL_PASSWORD=tango
+    volumes:
+      - se-tangodb:/var/lib/mysql
+
+  databaseds:
+    image: nexus.engageska-portugal.pt/ska-docker/tango-cpp:latest
+    depends_on:
+      - tangodb
+    network_mode: ${NETWORK_MODE}
+    container_name: ${CONTAINER_NAME_PREFIX}databaseds
+    environment:
+      - MYSQL_HOST=${MYSQL_HOST}
+      - MYSQL_DATABASE=tango
+      - MYSQL_USER=tango
+      - MYSQL_PASSWORD=tango
+      - TANGO_HOST=${TANGO_HOST}
+    entrypoint:
+      - /usr/local/bin/wait-for-it.sh
+      - ${MYSQL_HOST}
+      - --timeout=70
+      - --strict
+      - --
+      - /usr/local/bin/DataBaseds
+      - "2"
+      - -ORBendPoint
+      - giop:tcp::10000
+

Makefile → docker/test-harness/Makefile

@@ -7,21 +7,25 @@ SHELL = /bin/bash
 
 all: test lint
 
+# wait for the device to be available before beginning the test
+# A temporary volume is mounted at /build when 'make test' is executing.
 # The following steps copy across useful output to this volume which can
 # then be extracted to form the CI summary for the test procedure.
 test:
-
-	 python setup.py test | tee ./build/setup_py_test.stdout; \
-	 mv coverage.xml ./build/reports/code-coverage.xml;
-
-# The following steps copy across useful output to this volume which can
-# then be extracted to form the CI summary for the test procedure.
+	# option -k 'dir_name' excludes 'dir_name' contents 
+	cd /app && python setup.py test | tee setup_py_test.stdout
+	mkdir -p /build/reports && \
+        if [ -d /build ]; then \
+                mv /app/setup_py_test.stdout /build/csp-lmc-subelement-setup-test.stdout; \
+                mv /app/htmlcov /build/csp-lmc-subelement_htmlcov; \
+                mv /app/build/reports/csp-lmc-subelement-unit-tests.xml /build/reports; \
+                mv /app/coverage.xml /build; \
+        fi;
 lint:
-
-	# FIXME pylint needs to run twice since there is no way go gather the text and junit xml output at the same time
 	pip3 install pylint2junit; \
-	pylint --output-format=parseable ska_python_skeleton | tee ./build/code_analysis.stdout; \
-	pylint --output-format=pylint2junit.JunitReporter ska_python_skeleton > ./build/reports/linting.xml;
-
+        mkdir -p /build/reports; \
+        cd /app && pylint --output-format=parseable cspse/lmc | tee /build/csp-lmc-subelement-code-analysis.stdout; \
+        cd /app && pylint --output-format=pylint2junit.JunitReporter cspse/lmc > /build/reports/csp-lmc-subelement-linting.xml;
 
 .PHONY: all test lint
+

docker/test-harness/README.md

+This directory is uploaded to the container when 'make test' is executed. Files
+in this directory will be found inside /build once uploaded to the container.
+

docs/src/conf.py

@@ -17,11 +17,17 @@
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 #
+autodoc_mock_imports = ['PyTango', 'tango', 'tango.server','run', 'command',
+                 'future', 'future.utils', 'logging', 'logging.handlers', 'ska',
+                 'ska.base', 'ska.base.control_model', 'SKAMaster', 'SKASubarray','numpy', 
+                 'csp_lmc_common'
+                 ]
+autodoc_member_order = 'bysource'
+
 import os
 import sys
-sys.path.insert(0, os.path.abspath('../..'))
+sys.path.insert(0, os.path.abspath('../../'))
 import sphinx_rtd_theme
-
 def setup(app):
     app.add_stylesheet('css/custom.css')
     app.add_javascript('js/github.js')
@@ -45,7 +51,8 @@ extensions = ['sphinx.ext.autodoc',
     'sphinx.ext.ifconfig',
     'sphinx.ext.viewcode',
     'sphinx.ext.githubpages',
-              'recommonmark']
+    'recommonmark'
+    ]
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
@@ -60,9 +67,9 @@ source_suffix = ['.rst', '.md']
 master_doc = 'index'
 
 # General information about the project.
-project = 'developer.skatelescope.org'
-copyright = '2018, SKA Organization'
-author = 'Marco Bartolini'
+project = 'Csp:LMC Subelement Classes'
+copyright = '2020, INAF-SKA Organization'
+author = 'C.Baffa, E.Giani'
 
 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the
@@ -172,8 +179,8 @@ latex_elements = {
 # (source start file, target name, title,
 #  author, documentclass [howto, manual, or own class]).
 latex_documents = [
-    (master_doc, 'developerskatelescopeorg.tex', 'developer.skatelescope.org Documentation',
-     'Marco Bartolini', 'manual'),
+        (master_doc, 'CspLMCSubElement.tex', 'CSP.LMC Subelement TANGO Classes Documentation',
+     'SKA Organization', 'manual'),
 ]
 
 
@@ -182,7 +189,7 @@ latex_documents = [
 # One entry per manual page. List of tuples
 # (source start file, name, description, authors, manual section).
 man_pages = [
-    (master_doc, 'developerskatelescopeorg', 'developer.skatelescope.org Documentation',
+    (master_doc, 'csplmc-subelement', 'CSP.LMC Subelement TANGO Classes Documentation',
      [author], 1)
 ]
 
@@ -193,8 +200,8 @@ man_pages = [
 # (source start file, target name, title, author,
 #  dir menu entry, description, category)
 texinfo_documents = [
-    (master_doc, 'developerskatelescopeorg', 'developer.skatelescope.org Documentation',
-     author, 'developerskatelescopeorg', 'One line description of project.',
+    (master_doc, 'CSP.LMC-SubElement', 'CSP.LMC Subelement TANGO Classes Documentation',
+     author, 'CSP.LMC-SubElement', 'One line description of project.',
      'Miscellaneous'),
 ]