Ideas for new PLUGINS in elektra.
Prefer using great frameworks to do that in short time with high quality.


## Python plugins

resolver/storage hybrid


## Lua plugin

value transformations



## OS abstraction ##

chdir/getcwd
chroot
hostname
uname also to set
ulimit

getenv plugin: for /env/current

Shell expansion
Environment expansion

noresolver: make it always return 0?


## logging

log all warnings that happened


# File based plugins #

Most plugins work on the KeySet. Other plugins might just work
on the file (in the value of the parentKey), including:
- gzip the file
- encrypt the file
- git add/git commit



# Helper plugins #

## rename ##

further renaming features

recode of names (using iconv)
Namecencode
nameutf8encode
namehexencode

arbitrary encodings via python/lua function


## fix Limitations ##

Idea: fix limitations of current plugins using other plugins

Needed for yajl+ini:
	rewrite of values in directories as subkeys

fstab:
	support for spaces and so on (with \040)


## add Limitations ##

allow to lock down configuration for guis


## Helper Library for storage plugins ##

Refactoring hosts + ini?

path plugin:
	allow random-generated parts in filenames
	check NFS, mounting properties


# Storage plugins #

## Generic ##

Build a generic storage plugin to replace dump:
- preserving order
- preserving comments
- human read+writeable


## JSON ##

* should handle cppcms config files
  http://cppcms.com/wikipp/en/page/cppcms_1x_config
* should handle openicc
  doc/standards/OpenICC_device_config_DB.json
  http://www.freedesktop.org/wiki/Specifications/icc_meta_tag_for_monitor_profiles

http://json-schema.org

## PRETTY ##

latex plugin to make pretty printed key/value pairs
dot/graphviz plugin to visualize graph of keys
html plugin (to show elektra config in html format)

literate data
	good for documentation
	everything is ignored except special introduced key/value pairs
	<start><key><seperator><value><terminator>
	needs two more special characters next to seperator, but is
	otherwise alike

## folders ##

filesys:
	reintroduce? (useful for 0.7 migrations)
	empty folder will not show up (only with %%dirdata)
	Dirdata richtig escapen
	Metadata reinschreiben
	remove keys the new way
	compare sets between current and desired state (set operations)
	could be default-storage plugin, integrated resolver for previous siutation

folder:
	make subdirectires folders and individual files
	(multi-resolver)


## library supported ##

https://github.com/toml-lang/toml

MacOSX plist files

http://rfc.zeromq.org/spec:4/ZPL
	https://github.com/zeromq/czmq/blob/master/src/zconfig.c

xpath plugin:
	on an xml file with an mapping from (reduced) xpath<->elektra keys

C/C++ serialization
	(ksNew(keyNew()))
	reading with llvm?

/etc/groups plugin

other file formats:
	xfree
	apache
	kde-ini
	gconf
	xml
	inittab

tcl:
	parse output?
	struct output!

compile lenses as plugin:
	+ no dependency at runtime
	+ may be faster?

xfconf file storage (see #47)

### binary

(may need orphan search or one file per plugin)

Kyoto Cabinet
unqlite
libconfig
eet
tdb.samba (using tdb_traverse)
sqlite




## ERROR ##

ignore_error plugin:
	transform error of a plugin to a warning

error handling control flags:
	When errors occur, a decision must be made if the plugin is important enough to
	stop the request completely or to go on with the other succeeding backends.
	The Pluggable Authentication Modules, also known as PAM,
	have developed a method to decide that well.
	PAM uses the three control flags `required`, `optional` and `sufficient` for that.
	Elektra does not support control flags.
	Instead, the position of the plugin defines the semantics on how to proceed.
	Plugins up to the commit plugin are `required` and later plugins are `optional`.
	The control flag `sufficient` would make sense when support for multiple storage
	plugins arises.



# Generic plugins #

Shell bindings: exec program and send dump format to it and receive one
can use #!kdb shell
or also some other scripts, e.g. git commit





# FILTER Plugins #

compression:
	zlib
	snappy

Forensic Logging:
	environment, pid, gid, uid
	timezone + in ms
	IP

filter: key
	metadata löschen
	hidden keys


advanced:
	hash sum
	signing
	encrypting
	binding

base64 encoding (and reverse)

libgcryt (LGPL, GPLv3?) used by cppcms

infer links between configuration (user/a=@user/b)
	and copy values


## RESTRICTIONs ##

restrict plugins to be written to
plugin: readonly (do not allow to be written to backend)


## METAREWRITE plugin ##

meta data rewriter into value
replaces NULL plugin

binary, type and any other meta data
encoded into string value
also encode null value (only possible in binary)
and binary (including encoding of null bytes and terminating null?)


## TYPE ##

struct: checker
	cannot be deeply nested
	don't throw char*
	check arrays

Checkers:
	check if hosts are reachable (e.g. by ping)

units-of-measurements (e.g. m, kg, A, ..)


# RESOLVER #

## split up ##

nosync+nolock variants

resolver-debug for tests
	concurrency: sets breakpoints within (done!)
	folder: write into folders controlled by env
resolver-xdg-passwd
	use passwd with XDG overides
resolver-xdg
	use $HOME with XDG overides
resolver-hardcoded
	(just use KDB_DB_* for single users embedded systems)
	done! (needs better docu)
resolver-passwd
	done! (needs better docu)
resolver-posix
	using HOME def. in posix: http://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap08.html
	using USER as fallback? not def. in posix, but often in shells


## improvements ##

create new files securely:
	fdopen(open("/tmp", O_TMPFILE ` O_EXCL ` O_RDWR, 0600), "rw");
	linkat(AT_FDCWD, "/proc/self/fd/??", AT_FDCWD, pathname, AT_SYMLINK_FOLLOW)
	pass "/proc/self/fd/??" to plugins

better error messages:
	#9 (when opening file failed)
	resolver in kdbError should correct error message
	the storage plugins do not have to state reason then

make explicit state variable (for both user+system) or return fd!=-1

resolver:
	configureable fsync




# Improve the plugin system itself #

## VERSION IN CONTRACT ##

Configure file for plugin.h
with defines for verbosity, configuration, version numbers + contract in
static inline function
with licence, author in contract
Everything also in constants/build_info
add dependencies in contract?

## Contract on Plugin configuration ##

do not only depend on "provides" but on configuration another plugin
provides. This option would allow to add multiple plugins with same
functionality, but only one is configured to do it actually.
* type checker
* encoding

Contract-Checker (linking against libtools) should be plugin too
	makes sure only valid backend configurations are written

More information about Plugins:
	which elektra version built again
	compiler
	determine which plugins in all variants

## Refinement and Inclusion ##

The behavior of a plugin can be specialised and extended through
contract refinement and a contract inclusion mechanism.
They allow us to create new contracts using previously defined contracts.
So the contract language can be viewed as a plugin interconnection
language.

## Metadata ##

Plugins should state which metadata they can handle and which they can interpret or generate

## Changing Features at runtime

Plugins, that take care of character escaping and encoding,
are needed so that limited
storage plugins work properly.
Removing them at run time is not a good idea.
But other plugins that provide validation, notification and
logging may be turned off.
This could be done
with information passed to the respective plugin.

Another solution works
completely without the cooperation of plugins, by
removing the plugin from the chain of execution entirely.
For example,
if a notification daemon crashes, the affected plugins could be removed
until the service is up again.

We need to handle different situations for plugins.
Some plugins are not
allowed to be removed because they are required.
Some must stay because the administrator forces a specific policy.
Other plugins, however, can be removed.

## Multiple Storage

From high-availability systems a challenging requirement arises.
How is it possible to make configuration storage redundant?

To achieve that, every backend needs to write out configuration multiple
times.
Such backends will achieve a live-backup for configuration.
The usage of different storage plugins writing to different machines
even increases the availability of the configuration.

To be more failure tolerant, we also have to improve the reading of
configuration.
Instead of stopping when the first storage plugin yields an error, the
next one should be tried instead.
We need a strict policy to ensure that
these storage plugins always set an error if something goes wrong.



## Transactions using Global Plugins

We will see in the following examples that some tasks
in `kdbSet()`
have to happen respectively before, or after any other event occurs.
A way to achieve this is by a special backend.
The advantage of this approach is that plugins can also be reused for
these tasks.
These plugins will be referred to as \intro{global plugins} in
this section.

## Atomic Notification

Until now,
each backend has its own notification plugin.
This means, that for every backend that commits, the applications will
be notified.
Listeners to these events, however, want to know when every
backend during a `kdbSet()` transaction is settled.
A global plugin can notify that a sequence of events is finished.
Only then is it the time for other programs to update their
configuration.

## Journalling

When a program or computer crashes, committing may abort unexpectedly. If
some backends have already committed, while others have not, an unsatisfying
situation arises. The premise that `kdbSet()` is executed fully or not
is broken. But the program does not exist anymore and no further actions are
possible.
Journalling provides a way to fix the problem when the next `kdbSet()` by
another process is executed.

It works by writing a journal log instead of directly moving files. Once
the log is completely written and flushed to disk, a global plugin does
the actual renaming of the files.
When the program crashes in the first phase while writing the log, this
is no problem because no configuration file has been changed up to that
point.
The unfinished log file has to be discarded.
On the other hand,
the program can replay the finished journal whenever the applications
aborts at a later time.
No intermediate state without knowing what needs to be done
will occur.


## plugin commands ##

metadata that is evaluated by plugins

metadata checker plugin: determines which metadata is present

selective commit phases:
	define phases, minimum:
	commit
	check

norecursive:
	only fetch a single plugin without recursion


## DISTRIBUTOR BACKENDS ##

can be used as default using symlinks

1.st level citizen
	allow resolver only (one plugin for backend to be able to hierarchically stack backends)
	share modules globally (to allow loading of module within module)

mount: current implementation
mount-mpi: current implementation, but multithreaded
filesystem: implementation compatible with 0.7
lazy-mount
bind-mount (using override)
owner support
user-mount

do not hardcode 10 plugins (rationale?)


## DEFAULT PLUGINS ##

default plugin handling: resolver/storage

policy how default storage & resolver plugin are named
resolve the symlink and load the module with correct name

is meta data in default plugin needed?
	add rewrite plugins?

default storage plugin:
	changeable
	nickel with meta in [] sections?

DEFAULT_BACKEND ->
	DEFAULT_STORAGE_PLUGIN
	DEFAULT_RESOLVER_PLUGIN
	(+ make it work)
	good out-of-the-box behaviour

## CPPPLUGIN ##

cppplugin:
	make a nice and easy cpp solution for plugins
	(integration into cpp binding)
	assemble C symbols with macros
	keep C->C++ problems at one place to be included

C++: (FINISH cppplugin)
	how to write backends in C++
	Exceptions propagations
	(C++ Exceptions to error codes)
