Flatisfy ======== Flatisfy is your new companion to ease your search of a new housing :) It uses [Weboob](http://weboob.org/) to get all the housing posts on most of the websites offering housings posts, and then offers a bunch of pipelines to filter and deduplicate the fetched housings. It can be used as a command-line utility, but also exposes a web API and visualisation, to browse through the results. _Note_: It is targeted at French users (due to the currently supported websites), and in particular at people living close to Paris, as I developped it for my personal use, and am currently living in Paris :) Any feedback and merge requests to better support other countries / cities are more than welcome! _Note_: In this repository and across the code, I am using the name "flat". I use it as a placeholder for "housing" and consider both are interchangeable. This code is not restricted to handling flats only! ## Getting started 1. Clone the repository. 2. Install required Python modules: `pip install -r requirements.txt`. 3. Init a configuration file: `python -m flatisfy init-config > config.json`. Edit it according to your needs (see below). 4. Build the required data files: `python -m flatisfy build-data --config config.json`. 5. Use it to `fetch` (and output a filtered JSON list of flats) or `import` (into an SQLite database, for the web visualization) a list of flats matching your criteria. 6. Use `python -m flatisfy serve --config config.json` to serve the web app. ## Configuration List of configuration options: * `data_directory` is the directory in which you want data files to be stored. `null` is the default value and means default `XDG` location (typically `~/.local/share/flatisfy/`) * `max_entries` is the maximum number of entries to fetch **per Weboob backend** (that is per housing website). * `passes` is the number of passes to run on the data. First pass is a basic filtering and using only the informations from the housings list page. Second pass loads any possible information about the filtered flats and does better filtering. * `queries` is a list of queries defined in `flatboob` that should be fetched. * `database` is an SQLAlchemy URI to a database file. Defaults to `null` which means that it will store the database in the default location, in `data_directory`. * `navitia_api_key` is an API token for [Navitia](https://www.navitia.io/) which is required to compute travel times. ### Constraints You can specify constraints, under the `constraints` key. The available constraints are: * `area` (in m²), `bedrooms`, `cost` (in currency unit), `rooms`: this is a tuple of `(min, max)` values, defining an interval in which the value should lie. A `null` value means that any value is within this bound. * `postal_codes` is a list of allowed postal codes. You should include any postal code you want, and especially the postal codes close to the precise location you want. You MUST provide some postal codes. * `time_to` is a dictionary of places to compute travel time to them. Typically, ``` "time_to": { "foobar": { "gps": [LAT, LNG], "time": [min, max] } } ``` means that the housings must be between the `min` and `max` bounds (possibly `null`) from the place identified by the GPS coordinates `LAT` and `LNG` (latitude and longitude), and we call this place `foobar` in human-readable form. Beware that `time` constraints are in **seconds**. ## OpenData I am using the following datasets, available under `flatisfy/data_files`, which covers Paris. If you want to run the script using some other location, you might have to change these files by matching datasets. * [LaPoste Hexasmal](https://datanova.legroupe.laposte.fr/explore/dataset/laposte_hexasmal/?disjunctive.code_commune_insee&disjunctive.nom_de_la_commune&disjunctive.code_postal&disjunctive.libell_d_acheminement&disjunctive.ligne_5) for the list of cities and postal codes in France. * [RATP stations](https://data.ratp.fr/explore/dataset/positions-geographiques-des-stations-du-reseau-ratp/table/?disjunctive.stop_name&disjunctive.code_postal&disjunctive.departement) for the list of subway stations with their positions in Paris and nearby areas. Both datasets are licensed under the Open Data Commons Open Database License (ODbL): https://opendatacommons.org/licenses/odbl/. ## License The content of this repository is licensed under an MIT license, unless explicitly mentionned otherwise. ## Contributing See the `CONTRIBUTING.md` file for more infos. ## Thanks * [Weboob](http://weboob.org/) * The OpenData providers listed above! * Navitia for their really cool public transportation API. * A lots of Python modules, required for this script (see `requirements.txt`). * [Kresus](https://framagit.org/bnjbvr/kresus) which gave me part of the original idea (at least proved me such software based on scraping can achieve a high quality level :)