andpp/mopidy
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Merge branch 'v1.1.x' into develop

Browse Source
This commit is contained in:
Stein Magnus Jodal 2015-08-23 17:27:24 +02:00
commit 9033bebf1a
34 changed files with 194 additions and 89 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/api/js.rst
View File

@ -256,7 +256,7 @@ chain. The function will be called with the error object as the only argument:
.. code-block:: js
mopidy.playback.getCurrentTrack()
.catch(console.error.bind(console));
.catch(console.error.bind(console))
.done(printCurrentTrack);
You can also register the error handler at the end of the promise chain by

59
docs/changelog.rst
View File

@ -4,6 +4,65 @@ Changelog
This changelog is used to track all major changes to Mopidy.
v1.1.1 (UNRELEASED)
===================
Bug fix release.
- Core: Make :meth:`mopidy.core.LibraryController.refresh` work for all
backends with a library provider. Previously, it wrongly worked for all
backends with a playlists provider. (Fixes: :issue:`1257`)
- Core: Respect :confval:`core/cache_dir` and :confval:`core/data_dir` config
values added in 1.1.0 when creating the dirs Mopidy need to store data. This
should not change the behavior for desktop users running Mopidy. When running
Mopidy as a system service installed from a package which sets the core dir
configs properly (e.g. Debian and Arch packages), this fix avoids the
creation of a couple of directories that should not be used, typically
:file:`/var/lib/mopidy/.local` and :file:`/var/lib/mopidy/.cache`. (Fixes:
:issue:`1259`, PR: :issue:`1266`)
- Local: Deprecate :confval:`local/data_dir` and respect
:confval:`core/data_dir` instead. This does not change the defaults for
desktop users, only system services installed from packages that properly set
:confval:`core/data_dir`, like the Debian and Arch packages. (Fixes:
:issue:`1259`, PR: :issue:`1266`)
- M3U: Changed default for the :confval:`m3u/playlists_dir` from
``$XDG_DATA_DIR/mopidy/m3u`` to unset, which now means the extension's data
dir. This does not change the defaults for desktop users, only system
services installed from packages that properly set :confval:`core/data_dir`,
like the Debian and Arch pakages. (Fixes: :issue:`1259`, PR: :issue:`1266`)
- Stream: If "file" is present in the :confval:`stream/protocols` config value
and the :ref:`ext-file` extension is enabled, we exited with an error because
two extensions claimed the same URI scheme. We now log a warning recommending
to remove "file" from the :confval:`stream/protocols` config, and then
proceed startup. (Fixes: :issue:`1248`, PR: :issue:`1254`)
- Stream: Fix bug in new playlist parser. A non-ASCII char in an urilist
comment would cause a crash while parsing due to comparision of a non-ASCII
bytestring with a Unicode string. (Fixes: :issue:`1265`)
- File: Adjust log levels when failing to expand ``$XDG_MUSIC_DIR`` into a real
path. This usually happens when running Mopidy as a system service, and thus
with a limited set of environment variables. (Fixes: :issue:`1249`, PR:
:issue:`1255`)
- File: When browsing files, we no longer scan the files to check if they're
playable. This makes browsing of the file hierarchy instant for HTTP clients,
which do no scanning of the files' metadata, and a bit faster for MPD
clients, which no longer scan the files twice. (Fixes: :issue:`1260`, PR:
:issue:`1261`)
- Audio: Fix timeout handling in scanner. This regression caused timeouts to
expire before it should, causing scans to fail.
- Audio: Update scanner to emit MIME type instead of an error when missing a
plugin.
v1.1.0 (2015-08-09)
===================

12
docs/config.rst
View File

@ -65,6 +65,13 @@ Core configuration
Path to base directory for storing cached data.
Mopidy and extensions will use this path to cache data that can safely be
thrown away.
If your system is running from an SD card, it can help avoid wear and
corruption of your SD card by pointing this config to another location. If
you have enough RAM, a tmpfs might be a good choice.
When running Mopidy as a regular user, this should usually be
``$XDG_CACHE_DIR/mopidy``, i.e. :file:`~/.cache/mopidy`.
@ -85,6 +92,11 @@ Core configuration
Path to base directory for persistent data files.
Mopidy and extensions will use this path to store data that cannot be
be thrown away and reproduced without some effort. Examples include
Mopidy-Local's index of your media library and Mopidy-M3U's stored
playlists.
When running Mopidy as a regular user, this should usually be
``$XDG_DATA_DIR/mopidy``, i.e. :file:`~/.local/share/mopidy`.

17
docs/debian.rst
View File

@ -113,12 +113,17 @@ from a regular Mopidy setup you'll want to know about.
sudo service mopidy status
- Mopidy installed from a Debian package can use both Mopidy extensions
installed both from Debian packages and extensions installed with pip.
- Mopidy installed from a Debian package can use Mopidy extensions installed
both from Debian packages and with pip. This has always been the case.
The other way around does not work: Mopidy installed with pip can use
extensions installed with pip, but not extensions installed from a Debian
package. This is because the Debian packages install extensions into
Mopidy installed with pip can use extensions installed with pip, but
not extensions installed from a Debian package released before August 2015.
This is because the Debian packages used to install extensions into
:file:`/usr/share/mopidy` which is normally not on your ``PYTHONPATH``.
Thus, your pip-installed Mopidy will not find the Debian package-installed
Thus, your pip-installed Mopidy would not find the Debian package-installed
extensions.
In August 2015, all Mopidy extension Debian packages was modified to install
into :file:`/usr/lib/python2.7/dist-packages`, like any other Python Debian
package. Thus, Mopidy installed with pip can now use extensions installed
from Debian.

4
docs/ext/local.rst
View File

@ -47,8 +47,8 @@ active at a time.
To create a new library provider you must create class that implements the
:class:`mopidy.local.Library` interface and install it in the extension
registry under ``local:library``. Any data that the library needs to store on
disc should be stored in :confval:`local/data_dir` using the library name as
part of the filename or directory to avoid any conflicts.
disc should be stored in the extension's data dir, as returned by
:meth:`~mopidy.ext.Extension.get_data_dir`.
Configuration

3
docs/ext/m3u.rst
View File

@ -52,4 +52,5 @@ See :ref:`config` for general help on configuring Mopidy.
.. confval:: m3u/playlists_dir
Path to directory with M3U files.
Path to directory with M3U files. Unset by default, in which case the
extension's data dir is used to store playlists.

20
mopidy/__main__.py
View File

@ -75,15 +75,16 @@ def main():
args = root_cmd.parse(mopidy_args)
create_file_structures_and_config(args, extensions_data)
check_old_locations()
config, config_errors = config_lib.load(
args.config_files,
[d.config_schema for d in extensions_data],
[d.config_defaults for d in extensions_data],
args.config_overrides)
create_core_dirs(config)
create_initial_config_file(args, extensions_data)
check_old_locations()
verbosity_level = args.base_verbosity_level
if args.verbosity_level:
verbosity_level += args.verbosity_level
@ -166,12 +167,17 @@ def main():
raise
def create_file_structures_and_config(args, extensions_data):
path.get_or_create_dir(b'$XDG_DATA_DIR/mopidy')
path.get_or_create_dir(b'$XDG_CONFIG_DIR/mopidy')
def create_core_dirs(config):
path.get_or_create_dir(config['core']['cache_dir'])
path.get_or_create_dir(config['core']['config_dir'])
path.get_or_create_dir(config['core']['data_dir'])
def create_initial_config_file(args, extensions_data):
"""Initialize whatever the last config file is with defaults"""
# Initialize whatever the last config file is with defaults
config_file = args.config_files[-1]
if os.path.exists(path.expand_path(config_file)):
return

19
mopidy/audio/scan.py
View File

@ -12,8 +12,6 @@ from mopidy import exceptions
from mopidy.audio import utils
from mopidy.internal import encoding
_missing_plugin_desc = gst.pbutils.missing_plugin_message_get_description
_Result = collections.namedtuple(
'Result', ('uri', 'tags', 'duration', 'seekable', 'mime', 'playable'))
@ -134,12 +132,12 @@ def _process(pipeline, timeout_ms):
clock = pipeline.get_clock()
bus = pipeline.get_bus()
timeout = timeout_ms * gst.MSECOND
tags, mime, have_audio, missing_description = {}, None, False, None
tags, mime, have_audio, missing_message = {}, None, False, None
types = (gst.MESSAGE_ELEMENT | gst.MESSAGE_APPLICATION | gst.MESSAGE_ERROR
| gst.MESSAGE_EOS | gst.MESSAGE_ASYNC_DONE | gst.MESSAGE_TAG)
start = clock.get_time()
previous = clock.get_time()
while timeout > 0:
message = bus.timed_pop_filtered(timeout, types)
@ -147,8 +145,7 @@ def _process(pipeline, timeout_ms):
break
elif message.type == gst.MESSAGE_ELEMENT:
if gst.pbutils.is_missing_plugin_message(message):
missing_description = encoding.locale_decode(
_missing_plugin_desc(message))
missing_message = message
elif message.type == gst.MESSAGE_APPLICATION:
if message.structure.get_name() == 'have-type':
mime = message.structure['caps'].get_name()
@ -158,8 +155,10 @@ def _process(pipeline, timeout_ms):
have_audio = True
elif message.type == gst.MESSAGE_ERROR:
error = encoding.locale_decode(message.parse_error()[0])
if missing_description:
error = '%s (%s)' % (missing_description, error)
if missing_message and not mime:
caps = missing_message.structure['detail']
mime = caps.get_structure(0).get_name()
return tags, mime, have_audio
raise exceptions.ScannerError(error)
elif message.type == gst.MESSAGE_EOS:
return tags, mime, have_audio
@ -171,7 +170,9 @@ def _process(pipeline, timeout_ms):
# Note that this will only keep the last tag.
tags.update(utils.convert_taglist(taglist))
timeout -= clock.get_time() - start
now = clock.get_time()
timeout -= now - previous
previous = now
raise exceptions.ScannerError('Timeout after %dms' % timeout_ms)

2
mopidy/core/actor.py
View File

@ -161,7 +161,7 @@ class Backends(list):
for scheme in b.uri_schemes.get():
assert scheme not in backends_by_scheme, (
'Cannot add URI scheme %s for %s, '
'Cannot add URI scheme "%s" for %s, '
'it is already handled by %s'
) % (scheme, name(b), name(backends_by_scheme[scheme]))
backends_by_scheme[scheme] = b

2
mopidy/core/library.py
View File

@ -255,7 +255,7 @@ class LibraryController(object):
backends = {}
uri_scheme = urlparse.urlparse(uri).scheme if uri else None
for backend_scheme, backend in self.backends.with_playlists.items():
for backend_scheme, backend in self.backends.with_library.items():
backends.setdefault(backend, set()).add(backend_scheme)
for backend, backend_schemes in backends.items():

4
mopidy/ext.py
View File

@ -63,6 +63,8 @@ class Extension(object):
def get_cache_dir(self, config):
"""Get or create cache directory for the extension.
Use this directory to cache data that can safely be thrown away.
:param config: the Mopidy config object
:return: string
"""
@ -87,6 +89,8 @@ class Extension(object):
def get_data_dir(self, config):
"""Get or create data directory for the extension.
Use this directory to store data that should be persistent.
:param config: the Mopidy config object
:returns: string
"""

25
mopidy/file/library.py
View File

@ -71,7 +71,7 @@ class FileLibraryProvider(backend.LibraryProvider):
name = dir_entry.decode(FS_ENCODING, 'replace')
if os.path.isdir(child_path):
result.append(models.Ref.directory(name=name, uri=uri))
elif os.path.isfile(child_path) and self._is_audio_file(uri):
elif os.path.isfile(child_path):
result.append(models.Ref.track(name=name, uri=uri))
result.sort(key=operator.attrgetter('name'))
@ -108,12 +108,15 @@ class FileLibraryProvider(backend.LibraryProvider):
media_dir_split[0].encode(FS_ENCODING))
if not local_path:
logger.warning('Failed expanding path (%s) from'
'file/media_dirs config value.',
media_dir_split[0])
logger.debug(
'Failed expanding path (%s) from file/media_dirs config '
'value.',
media_dir_split[0])
continue
elif not os.path.isdir(local_path):
logger.warning('%s is not a directory', local_path)
logger.warning(
'%s is not a directory. Please create the directory or '
'update the file/media_dirs config value.', local_path)
continue
media_dir['path'] = local_path
@ -131,18 +134,6 @@ class FileLibraryProvider(backend.LibraryProvider):
name=media_dir['name'],
uri=path.path_to_uri(media_dir['path']))
def _is_audio_file(self, uri):
try:
result = self._scanner.scan(uri)
if result.playable:
logger.debug('Playable file: %s', result.uri)
else:
logger.debug('Unplayable file: %s (not audio)', result.uri)
return result.playable
except exceptions.ScannerError as e:
logger.debug('Unplayable file: %s (%s)', uri, e)
return False
def _is_in_basedir(self, local_path):
return any(
path.is_path_inside_base_dir(local_path, media_dir['path'])

2
mopidy/internal/playlists.py
View File

@ -122,7 +122,7 @@ def parse_asx(data):
def parse_urilist(data):
result = []
for line in data.splitlines():
if not line.strip() or line.startswith('#'):
if not line.strip() or line.startswith(b'#'):
continue
try:
validation.check_uri(line)

2
mopidy/local/__init__.py
View File

@ -23,7 +23,7 @@ class Extension(ext.Extension):
schema = super(Extension, self).get_config_schema()
schema['library'] = config.String()
schema['media_dir'] = config.Path()
schema['data_dir'] = config.Path()
schema['data_dir'] = config.Deprecated()
schema['playlists_dir'] = config.Deprecated()
schema['tag_cache_file'] = config.Deprecated()
schema['scan_timeout'] = config.Integer(

1
mopidy/local/ext.conf
View File

@ -2,7 +2,6 @@
enabled = true
library = json
media_dir = $XDG_MUSIC_DIR
data_dir = $XDG_DATA_DIR/mopidy/local
scan_timeout = 1000
scan_flush_threshold = 1000
scan_follow_symlinks = false

4
mopidy/local/json.py
View File

@ -12,7 +12,7 @@ import tempfile
import mopidy
from mopidy import compat, local, models
from mopidy.internal import encoding, timer
from mopidy.local import search, storage, translator
from mopidy.local import Extension, search, storage, translator
logger = logging.getLogger(__name__)
@ -116,7 +116,7 @@ class JsonLibrary(local.Library):
self._browse_cache = None
self._media_dir = config['local']['media_dir']
self._json_file = os.path.join(
config['local']['data_dir'], b'library.json.gz')
Extension().get_data_dir(config), b'library.json.gz')
storage.check_dirs_and_files(config)

9
mopidy/local/storage.py
View File

@ -3,8 +3,6 @@ from __future__ import absolute_import, unicode_literals
import logging
import os
from mopidy.internal import encoding, path
logger = logging.getLogger(__name__)
@ -13,10 +11,3 @@ def check_dirs_and_files(config):
logger.warning(
'Local media dir %s does not exist.' %
config['local']['media_dir'])
try:
path.get_or_create_dir(config['local']['data_dir'])
except EnvironmentError as error:
logger.warning(
'Could not create local data dir: %s',
encoding.locale_decode(error))

2
mopidy/m3u/__init__.py
View File

@ -21,7 +21,7 @@ class Extension(ext.Extension):
def get_config_schema(self):
schema = super(Extension, self).get_config_schema()
schema['playlists_dir'] = config.Path()
schema['playlists_dir'] = config.Path(optional=True)
return schema
def setup(self, registry):

18
mopidy/m3u/actor.py
View File

@ -4,7 +4,7 @@ import logging
import pykka
from mopidy import backend
from mopidy import backend, m3u
from mopidy.internal import encoding, path
from mopidy.m3u.library import M3ULibraryProvider
from mopidy.m3u.playlists import M3UPlaylistsProvider
@ -21,12 +21,16 @@ class M3UBackend(pykka.ThreadingActor, backend.Backend):
self._config = config
try:
path.get_or_create_dir(config['m3u']['playlists_dir'])
except EnvironmentError as error:
logger.warning(
'Could not create M3U playlists dir: %s',
encoding.locale_decode(error))
if config['m3u']['playlists_dir'] is not None:
self._playlists_dir = config['m3u']['playlists_dir']
try:
path.get_or_create_dir(self._playlists_dir)
except EnvironmentError as error:
logger.warning(
'Could not create M3U playlists dir: %s',
encoding.locale_decode(error))
else:
self._playlists_dir = m3u.Extension().get_data_dir(config)
self.playlists = M3UPlaylistsProvider(backend=self)
self.library = M3ULibraryProvider(backend=self)

2
mopidy/m3u/ext.conf
View File

@ -1,3 +1,3 @@
[m3u]
enabled = true
playlists_dir = $XDG_DATA_DIR/mopidy/m3u
playlists_dir =

2
mopidy/m3u/playlists.py
View File

@ -23,7 +23,7 @@ class M3UPlaylistsProvider(backend.PlaylistsProvider):
def __init__(self, *args, **kwargs):
super(M3UPlaylistsProvider, self).__init__(*args, **kwargs)
self._playlists_dir = self.backend._config['m3u']['playlists_dir']
self._playlists_dir = self.backend._playlists_dir
self._playlists = {}
self.refresh()

6
mopidy/mpd/dispatcher.py
View File

@ -271,9 +271,9 @@ class MpdContext(object):
If ``lookup`` is true and the ``path`` is to a track, the returned
``data`` is a future which will contain the results from looking up
the URI with :meth:`mopidy.core.LibraryController.lookup` If ``lookup``
is false and the ``path`` is to a track, the returned ``data`` will be
a :class:`mopidy.models.Ref` for the track.
the URI with :meth:`mopidy.core.LibraryController.lookup`. If
``lookup`` is false and the ``path`` is to a track, the returned
``data`` will be a :class:`mopidy.models.Ref` for the track.
For all entries that are not tracks, the returned ``data`` will be
:class:`None`.

7
mopidy/stream/actor.py
View File

@ -36,6 +36,13 @@ class StreamBackend(pykka.ThreadingActor, backend.Backend):
self.uri_schemes = audio_lib.supported_uri_schemes(
config['stream']['protocols'])
if 'file' in self.uri_schemes and config['file']['enabled']:
logger.warning(
'The stream/protocols config value includes the "file" '
'protocol. "file" playback is now handled by Mopidy-File. '
'Please remove it from the stream/protocols config.')
self.uri_schemes -= {'file'}
class StreamLibraryProvider(backend.LibraryProvider):

18
tests/audio/test_scan.py
View File

@ -40,8 +40,11 @@ class ScannerTest(unittest.TestCase):
self.assertEqual(self.result[name].tags[key], value)
def check_if_missing_plugin(self):
if any(['missing a plug-in' in str(e) for e in self.errors.values()]):
raise unittest.SkipTest('Missing MP3 support?')
for path, result in self.result.items():
if not path.endswith('.mp3'):
continue
if not result.playable and result.mime == 'audio/mpeg':
raise unittest.SkipTest('Missing MP3 support?')
def test_tags_is_set(self):
self.scan(self.find('scanner/simple'))
@ -109,6 +112,17 @@ class ScannerTest(unittest.TestCase):
wav = path_to_data_dir('scanner/empty.wav')
self.assertEqual(self.result[wav].duration, 0)
def test_uri_list(self):
path = path_to_data_dir('scanner/playlist.m3u')
self.scan([path])
self.assertEqual(self.result[path].mime, 'text/uri-list')
def test_text_plain(self):
# GStreamer decode bin hardcodes bad handling of text plain :/
path = path_to_data_dir('scanner/plain.txt')
self.scan([path])
self.assertIn(path, self.errors)
@unittest.SkipTest
def test_song_without_time_is_handeled(self):
pass

3
tests/core/test_actor.py
View File

@ -37,7 +37,8 @@ class CoreActorTest(unittest.TestCase):
self.assertRaisesRegexp(
AssertionError,
'Cannot add URI scheme dummy1 for B2, it is already handled by B1',
'Cannot add URI scheme "dummy1" for B2, '
'it is already handled by B1',
Core, mixer=None, backends=[self.backend1, self.backend2])
def test_version(self):

6
tests/core/test_library.py
View File

@ -20,6 +20,7 @@ class BaseCoreLibraryTest(unittest.TestCase):
self.library1.get_images.return_value.get.return_value = {}
self.library1.root_directory.get.return_value = dummy1_root
self.backend1.library = self.library1
self.backend1.has_playlists.return_value.get.return_value = False
dummy2_root = Ref.directory(uri='dummy2:directory', name='dummy2')
self.backend2 = mock.Mock()
@ -29,13 +30,14 @@ class BaseCoreLibraryTest(unittest.TestCase):
self.library2.get_images.return_value.get.return_value = {}
self.library2.root_directory.get.return_value = dummy2_root
self.backend2.library = self.library2
self.backend2.has_playlists.return_value.get.return_value = False
# A backend without the optional library provider
self.backend3 = mock.Mock()
self.backend3.uri_schemes.get.return_value = ['dummy3']
self.backend3.actor_ref.actor_class.__name__ = 'DummyBackend3'
self.backend3.has_library().get.return_value = False
self.backend3.has_library_browse().get.return_value = False
self.backend3.has_library.return_value.get.return_value = False
self.backend3.has_library_browse.return_value.get.return_value = False
self.core = core.Core(mixer=None, backends=[
self.backend1, self.backend2, self.backend3])

0
tests/data/library.json.gz → tests/data/local/library.json.gz
View File

1
tests/data/scanner/plain.txt Normal file
View File

@ -0,0 +1 @@
Some plain text file with nothing special in it.

1
tests/data/scanner/playlist.m3u Normal file
View File

@ -0,0 +1 @@
http://example.com/

2
tests/internal/test_playlists.py
View File

@ -23,7 +23,7 @@ file:///tmp/baz
URILIST = b"""
file:///tmp/foo
# a comment
# a comment \xc5\xa7\xc5\x95
file:///tmp/bar
file:///tmp/baz

5
tests/local/test_json.py
View File

@ -45,10 +45,11 @@ class BrowseCacheTest(unittest.TestCase):
class JsonLibraryTest(unittest.TestCase):
config = {
'core': {
'data_dir': path_to_data_dir(''),
},
'local': {
'media_dir': path_to_data_dir(''),
'data_dir': path_to_data_dir(''),
'playlists_dir': b'',
'library': 'json',
},
}

18
tests/local/test_library.py
View File

@ -65,10 +65,11 @@ class LocalLibraryProviderTest(unittest.TestCase):
]
config = {
'core': {
'data_dir': path_to_data_dir(''),
},
'local': {
'media_dir': path_to_data_dir(''),
'data_dir': path_to_data_dir(''),
'playlists_dir': b'',
'library': 'json',
},
}
@ -105,11 +106,15 @@ class LocalLibraryProviderTest(unittest.TestCase):
tmpdir = tempfile.mkdtemp()
try:
tmplib = os.path.join(tmpdir, 'library.json.gz')
shutil.copy(path_to_data_dir('library.json.gz'), tmplib)
tmpdir_local = os.path.join(tmpdir, 'local')
shutil.copytree(path_to_data_dir('local'), tmpdir_local)
config = {'local': self.config['local'].copy()}
config['local']['data_dir'] = tmpdir
config = {
'core': {
'data_dir': tmpdir,
},
'local': self.config['local'],
}
backend = actor.LocalBackend(config=config, audio=None)
# Sanity check that value is in the library
@ -117,6 +122,7 @@ class LocalLibraryProviderTest(unittest.TestCase):
self.assertEqual(result, self.tracks[0:1])
# Clear and refresh.
tmplib = os.path.join(tmpdir_local, 'library.json.gz')
open(tmplib, 'w').close()
backend.library.refresh()

3
tests/local/test_playback.py
View File

@ -23,12 +23,11 @@ from tests.local import generate_song, populate_tracklist
class LocalPlaybackProviderTest(unittest.TestCase):
config = {
'core': {
'data_dir': path_to_data_dir(''),
'max_tracklist_length': 10000,
},
'local': {
'media_dir': path_to_data_dir(''),
'data_dir': path_to_data_dir(''),
'playlists_dir': b'',
'library': 'json',
}
}

2
tests/local/test_tracklist.py
View File

@ -18,11 +18,11 @@ from tests.local import generate_song, populate_tracklist
class LocalTracklistProviderTest(unittest.TestCase):
config = {
'core': {
'data_dir': path_to_data_dir(''),
'max_tracklist_length': 10000
},
'local': {
'media_dir': path_to_data_dir(''),
'data_dir': path_to_data_dir(''),
'playlists_dir': b'',
'library': 'json',
}