Rationale¶
Approaches to sockets and message passing¶
Whisker allows a multitude of clients in a great many languages – anything that can “speak” down a TCP/IP port, such as C++, Visual Basic, Perl, and Python.
Whisker uses two sockets, a main socket, through which the Whisker server can send events unprompted, and an immediate socket, used for sending commands/queries and receiving immediate replies with a one-to-one relationship between commands (client → server) and responses (server → client). Consequently, the client must deal with unpredictable events coming from the server. It might also have to deal with some sort of user interface (UI) code (and other faster things, like data storage).
C++¶
In C++/MFC, sockets get their own thread anyway, and the framework tries to hide this from you. So the GUI and sockets coexists fairly happily. Many Whisker tasks use C++, but it’s not the easiest thing in the world.
Perl¶
In Perl, I’ve used only a very basic approach with a manual message loop, like this:
# Enter a loop to listen to messages from the server.
while (chomp($line = <$MAINSOCK>)) {
if ($line =~ /^Ping/) {
# If the server has sent us a Ping, acknowledge it.
Send("PingAcknowledged");
} elsif ($line =~ /^Event: (.+)/) {
# The server has sent us an event.
$event = $1;
# Event handling for the behavioural task is dealt with here.
if ($event =~ /^EndOfTask$/) { last; } # Exit the while loop.
}
}
Python¶
In Python, I’ve used the following approaches:
Manual event loop¶
You can use base socket code, and poll the main socket for input regularly. Simple. But you can forget about simultaneous UI. Like this:
Non-threaded, event-driven¶
The Twisted library is great for this (https://twistedmatrix.com/). However:
- Bits of it, like Tkinter integration, still don’t support Python 3 fully (as of 2015-12-23), though this is not a major problem (it’s easy to hack in relevant bits of Python 3 support).
- Though it will integrate its event loop (reactor) with several GUI toolkits, e.g. http://twistedmatrix.com/documents/current/core/howto/choosing-reactor.html this can still fail; e.g. with Tkinter, if you open a system dialogue (such as the standard “Open File…” or “Save As…” dialogues), the Twisted reactor will stop and wait, which is no good. This is a much bigger problem. (More detail on this problem in my dev_notes.txt for the starfeeder project.)
- So one could use Twisted with no user interaction during the task.
It looks, from the task writer’s perspective, like this:
#!/usr/bin/env python
"""
whisker/test_twisted.py
===============================================================================
Copyright © 2011-2020 Rudolf Cardinal (rudolf@pobox.com).
This file is part of the Whisker Python client library.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
===============================================================================
**Command-line tool to test the Whisker Twisted client.**
"""
import argparse
import logging
from twisted.internet import reactor
from cardinal_pythonlib.logs import configure_logger_for_colour
from whisker.api import (
Pen,
PenStyle,
BrushStyle,
BrushHatchStyle,
Brush,
Rectangle,
)
from whisker.constants import DEFAULT_PORT
from whisker.twistedclient import WhiskerTwistedTask
log = logging.getLogger(__name__)
DEFAULT_DISPLAY_NUM = 0
DEFAULT_AUDIO_NUM = 0
DEFAULT_INPUT_LINE = 0
DEFAULT_OUTPUT_LINE = 64
DEFAULT_MEDIA_DIR = r"C:\Program Files\WhiskerControl\Server Test Media"
DEFAULT_BITMAP = "santa_fe.bmp"
DEFAULT_VIDEO = "mediaexample.wmv"
DEFAULT_WAV = "telephone.wav"
AUDIO = "audio"
DISPLAY = "display"
DOC = "doc"
VIDEO = "_video"
class MyWhiskerTwistedTask(WhiskerTwistedTask):
"""
Class deriving from :class:`whisker.twistedclient.WhiskerTwistedTask` to
demonstrate the Twisted Whisker client.
"""
def __init__(self,
display_num: int,
audio_num: int,
input_line: str,
output_line: str,
media_dir: str,
bitmap: str,
video: str,
wav: str) -> None:
super().__init__() # call base class init
self.display_num = display_num
self.audio_num = audio_num
self.input = input_line
self.output = output_line
self.media_dir = media_dir
self.bitmap = bitmap
self.video = video
self.wav = wav
# ... anything extra here
def fully_connected(self) -> None:
"""
Called when the server is fully connected. Set up the "task".
"""
print("SENDING SOME TEST/DEMONSTRATION COMMANDS")
self.whisker.get_network_latency_ms()
self.whisker.report_name("Whisker Twisted client prototype")
self.whisker.timestamps(True)
self.whisker.timer_set_event("TimerFired", 1000, 9)
self.whisker.timer_set_event("EndOfTask", 12000)
# ---------------------------------------------------------------------
# Audio
# ---------------------------------------------------------------------
# ---------------------------------------------------------------------
# Display
# ---------------------------------------------------------------------
bg_col = (0, 0, 100)
pen = Pen(width=3, colour=(255, 255, 150), style=PenStyle.solid)
brush = Brush(
colour=(255, 0, 0), bg_colour=(0, 255, 0),
opaque=True, style=BrushStyle.hatched,
hatch_style=BrushHatchStyle.bdiagonal)
self.whisker.claim_display(number=self.display_num, alias=DISPLAY)
display_size = self.whisker.display_get_size(DISPLAY)
log.info("display_size: {}".format(display_size))
self.whisker.display_scale_documents(DISPLAY, True)
self.whisker.display_create_document(DOC)
self.whisker.display_set_background_colour(DOC, bg_col)
self.whisker.display_blank(DISPLAY)
self.whisker.display_create_document("junk")
self.whisker.display_delete_document("junk")
self.whisker.display_show_document(DISPLAY, DOC)
with self.whisker.display_cache_wrapper(DOC):
self.whisker.display_add_obj_text(
DOC, "_text", (50, 50), "hello there!",
italic=True)
self.whisker.display_add_obj_line(
DOC, "_line", (25, 25), (200, 200), pen)
self.whisker.display_add_obj_arc(
DOC, "_arc",
Rectangle(left=100, top=100, width=200, height=200),
(25, 25), (200, 200), pen)
self.whisker.display_add_obj_bezier(
DOC, "_bezier",
(100, 100), (150, 100),
(150, 200), (100, 200),
pen)
self.whisker.display_add_obj_chord(
DOC, "_chord",
Rectangle(left=300, top=0, width=100, height=100),
(100, 150), (400, 175),
pen, brush)
self.whisker.display_add_obj_ellipse(
DOC, "_ellipse",
Rectangle(left=0, top=200, width=200, height=100),
pen, brush)
self.whisker.display_add_obj_pie(
DOC, "_pie",
Rectangle(left=0, top=300, width=200, height=100),
(10, 320), (180, 380),
pen, brush)
self.whisker.display_add_obj_polygon(
DOC, "_polygon",
[(400, 200), (450, 300), (400, 400), (300, 300)],
pen, brush, alternate=True)
self.whisker.display_add_obj_rectangle(
DOC, "_rectangle",
Rectangle(left=500, top=0, width=200, height=100),
pen, brush)
self.whisker.display_add_obj_roundrect(
DOC, "_roundrect",
Rectangle(left=500, top=200, width=100, height=200),
150, 250,
pen, brush)
self.whisker.display_add_obj_camcogquadpattern(
DOC, "_camcogquad",
(500, 400),
10, 10,
[1, 2, 4, 8, 16, 32, 64, 128],
[255, 254, 253, 252, 251, 250, 249, 248],
[1, 2, 3, 4, 5, 6, 7, 8],
[128, 64, 32, 16, 8, 4, 2, 1],
(255, 0, 0),
(0, 255, 0),
(0, 0, 255),
(255, 0, 255),
(100, 100, 100))
self.whisker.display_set_event(DOC, "_polygon", "poly_touched")
self.whisker.display_clear_event(DOC, "_polygon")
self.whisker.display_set_event(DOC, "_camcogquad", "play")
self.whisker.display_set_event(DOC, "_roundrect", "pause")
self.whisker.display_set_event(DOC, "_rectangle", "stop")
self.whisker.display_set_obj_event_transparency(DOC, "_rectangle",
False)
self.whisker.display_bring_to_front(DOC, "_chord")
self.whisker.display_send_to_back(DOC, "_chord")
self.whisker.display_keyboard_events(DOC)
self.whisker.display_event_coords(False)
self.whisker.display_set_audio_device(DISPLAY, AUDIO)
self.whisker.display_add_obj_video(DOC, VIDEO, (600, 0), self.video)
self.whisker.video_set_volume(DOC, VIDEO, 90)
self.whisker.video_timestamps(True)
video_dur_ms = self.whisker.video_get_duration_ms(DOC, VIDEO)
log.info("video_dur_ms: {}".format(video_dur_ms))
video_pos_ms = self.whisker.video_get_time_ms(DOC, VIDEO)
log.info("video_pos_ms: {}".format(video_pos_ms))
def incoming_event(self, event: str, timestamp: int = None) -> None:
"""
Responds to incoming events from Whisker.
"""
print("Event: {e} (timestamp {t})".format(e=event, t=timestamp))
if event == "EndOfTask":
# noinspection PyUnresolvedReferences
reactor.stop()
elif event == "play":
self.whisker.video_play(DOC, VIDEO)
elif event == "pause":
self.whisker.video_pause(DOC, VIDEO)
elif event == "stop":
self.whisker.video_stop(DOC, VIDEO)
elif event == "back":
self.whisker.video_seek_relative(DOC, VIDEO, -1000)
elif event == "forward":
self.whisker.video_seek_relative(DOC, VIDEO, 1000)
elif event == "start":
self.whisker.video_seek_absolute(DOC, VIDEO, 0)
def main() -> None:
"""
Command-line parser.
See ``--help`` for details.
"""
logging.basicConfig()
logging.getLogger("whisker").setLevel(logging.DEBUG)
configure_logger_for_colour(logging.getLogger()) # configure root logger
# print_report_on_all_logs()
parser = argparse.ArgumentParser(
description="Test Whisker Twisted client",
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
)
parser.add_argument(
'--server', default='localhost',
help="Server")
parser.add_argument(
'--port', default=DEFAULT_PORT, type=int,
help="Port")
parser.add_argument(
'--display_num', default=DEFAULT_DISPLAY_NUM, type=int,
help="Display number to use")
parser.add_argument(
'--audio_num', default=DEFAULT_AUDIO_NUM, type=int,
help="Audio device number to use")
parser.add_argument(
'--input', default=DEFAULT_INPUT_LINE, type=int,
help="Input line number to use")
parser.add_argument(
'--output', default=DEFAULT_OUTPUT_LINE, type=int,
help="Output line number to use")
parser.add_argument(
'--media_dir', default=DEFAULT_MEDIA_DIR, type=str,
help="Media directory to use")
parser.add_argument(
'--bitmap', default=DEFAULT_BITMAP, type=str,
help="Bitmap to use")
parser.add_argument(
'--video', default=DEFAULT_VIDEO, type=str,
help="Video to use")
parser.add_argument(
'--wav', default=DEFAULT_WAV, type=str,
help="WAV file to use")
args = parser.parse_args()
print("Module run explicitly. Running a Whisker test.")
w = MyWhiskerTwistedTask(
display_num=args.display_num,
audio_num=args.audio_num,
input_line=args.input,
output_line=args.output,
media_dir=args.media_dir,
bitmap=args.bitmap,
video=args.video,
wav=args.wav,
)
w.connect(args.server, args.port)
reactor.run()
if __name__ == '__main__':
main()
Multithreading¶
For multithreading we can use Qt (with the PySide bindings). In this approach, the Whisker task runs in separate threads from the UI. This works well, though is not without some complexity. The Qt interface is nice, and can be fairly fancy. You have to be careful with database access if using SQLite (which is not always happy in a multithreaded context).
Verdict for simple uses¶
Use Twisted and avoid any UI code while the task is running.
Database access¶
Database backend¶
There are distinct advantages to making SQLite the default, namely:
- It comes with everything (i.e. no installation required);
- Database can be copied around as single files.
On the downside, it doesn’t cope with multithreading/multiuser access quite as well as “bigger” databases like MySQL.
Users will want simple textfile storage as well.
Front end¶
The options for SQLite access include direct access:
and SQLAlchemy:
Getting fancier, it’s possible to manage database structure migrations with tools like Alembic (for SQLAlchemy), but this may be getting too complicated for the target end user.
However, the other very pleasantly simple front-end is dataset:
User interface¶
A GUI can consume a lot of programmer effort. Let’s keep this minimal or absent as the general rule; for more advanced coding, the coder can do his/her own thing (a suggestion: Qt).
Task configuration¶
Much of the GUI is usually about configuration. So let’s get rid of all that, because we’re aiming at very simple programming here. Let’s just put config in a simple structure like JSON or YAML, and have the user edit it separately.
JSON¶
An example program:
#!/usr/bin/env python
# json_example.py
import json
# Demo starting values
# config = {
# "session": 3,
# "stimulus_order": [3, 2, 1],
# "subject": "S1"
# }
# or, equivalently:
config = dict(
session=3,
stimulus_order=[3, 2, 1],
subject='S1'
)
# Save
with open("json_config_demo.json", "w") as outfile:
json.dump(config, outfile,
sort_keys=True, indent=4, separators=(',', ': '))
# Load
with open("json_config_demo.json") as infile:
config = json.load(infile)
The JSON looks like:
{
"session": 3,
"stimulus_order": [
3,
2,
1
],
"subject": "S1"
}
YAML with attrdict¶
This can be a bit fancier in terms of the object structure it can represent, a bit cleaner in terms of the simplicity of the config file, and safer in terms of security from dodgy config files.
Using an AttrDict allows a cleaner syntax for reading/writing the Python object.
#!/usr/bin/env python
# yaml_example.py
from attrdict import AttrDict
import yaml
# Demo starting values; lots of ways of creating an AttrDict
config = AttrDict()
config.session = 3
config.stimulus_order = [3, 2, 1]
config.subject = "S1"
# Save
with open("yaml_config_demo.yaml", "w") as outfile:
outfile.write(yaml.dump(config.copy())) # write a dict, not an AttrDict
# Load
with open("yaml_config_demo.yaml") as infile:
config = AttrDict(yaml.safe_load(infile))
The YAML looks like this:
session: 3
stimulus_order: [3, 2, 1]
subject: S1
A quick YAML tutorial¶
A key:value pair looks like:
key: value
A list looks like:
list:
- value1
- value2
# ...
A dictionary looks like:
dict:
key1: value1
key2: value2
# ...
Verdict for simple uses¶
Use YAML with AttrDict.
Package distribution¶
This should be via PyPI, so users can just do:
pip3 install whisker
from whisker import * # but better actual symbols than "*"!