phyks
/
flatisfy
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
291 Commits 8 Branches 0 Tags 15 MiB
Commit Graph

213 Commits

Author SHA1 Message Date
nicofrand 5b85ad6e59 Add unit tests 2018-01-17 08:43:25 +01:00
Lucas Verney 48835c0d83 Make "flatisfy" key optional in deep_duplicate computation 2018-01-12 14:48:10 +01:00
Lucas Verney 46457b014a Avoid too broad ignoring of flats when looking for only some postal codes in a city covered by multiple postal codes. Fix #110. 2018-01-10 20:01:22 +01:00
Lucas Verney 45c4eca775 Add a logging line after webserver is started. 
Also remove useless `debug` config option.
 2018-01-10 19:41:51 +01:00
Lucas Verney 47ce1ba7e0 Ignore duplicates by IDs, no need to even add the duplicate in db 2018-01-10 17:03:18 +01:00
Lucas Verney 35c88050df Better logging for station metadata detection 2018-01-10 16:46:30 +01:00
Lucas Verney 219b8e156e Fix a bug when throwing an exception with unicode characters from 
Weboob.

As reported by @nicofrand.
 2018-01-06 10:53:35 +01:00
Lucas Verney 3f30257b44 Fix spelling in deep_detect 2018-01-05 19:43:07 +01:00
Lucas Verney a6b193ae87 Fix missing config arg in deep_detect 2018-01-05 19:01:08 +01:00
Lucas Verney 83ff6be409 Better fix for #100 2017-12-30 19:30:32 +01:00
Lucas Verney 67d1299b15 Consider duplicates cannot belong to the same website, should fix #100. 2017-12-30 15:27:46 +01:00
Lucas Verney 7196add731 Lint code 2017-12-30 14:45:58 +01:00
Lucas Verney 1119bdbf0b Allow to define the threshold between a housing and a station 
Fix #101.
 2017-12-29 22:58:05 +01:00
Lucas Verney 49ebfe9111 Add an opendata endpoint in API 2017-12-13 14:38:05 +01:00
Lucas Verney a1846782ba Add support for PATCH method in CORS header 2017-12-08 15:47:59 +01:00
Lucas Verney 03d2ac8b80 Support sorting and pagination in API 2017-12-07 16:07:47 +01:00
Lucas Verney c3941bd70c Rebuild data on import command if required 2017-12-06 19:16:24 +01:00
Lucas Verney 0376cdf4e0 Fix a bug with Flat.from_dict 2017-12-06 19:03:25 +01:00
Lucas Verney 03d2ad6f7f Add CORS headers in the API response, add a debug option to get verbose debugging from Bottle webserver 2017-12-05 17:33:19 +01:00
Lucas Verney 8a74a79ac2 Update doc link in README.md 2017-12-05 15:17:03 +01:00
Lucas Verney daf16d88ba Sphinx-generated doc 2017-12-05 14:56:08 +01:00
Lucas Verney b285c270aa Cleaner API 2017-12-05 12:20:40 +01:00
Lucas Verney 9424f81959 Rework API code 
Factor flat post-processing in API and fix for #83. Always return JSON
even if an exception occurred.
 2017-12-04 22:01:46 +01:00
Lucas Verney aa0e1fd965 Use a single common data source for public transports stops 
Now makes use of Navitia opendata dumps to cover France. Fixes #65.
 2017-12-04 16:14:52 +01:00
Lucas Verney 376b327379 Drop support for entreparticuliers Weboob module 2017-12-03 22:26:48 +01:00
Lucas Verney 64f6e8886b Deduplicate the available backends list 2017-11-27 15:05:13 +01:00
Lucas Verney 702f58cf9e Fix JS lint 2017-11-27 10:42:22 +01:00
Lucas Verney 571b42f5b1 Check that postal codes are strings in config 2017-11-26 16:51:50 +01:00
Lucas Verney 4a65f52475 Do not store personal data by default, let user overload this behavior 
through config.

Fixes #80.
 2017-11-17 16:14:11 +01:00
Lucas Verney c936228726 Add an ICS feed of visits 
UI is minimalist and should be improved in the future. Fixes #40.
 2017-11-10 16:29:38 +01:00
Lucas Verney d6bee1dcb0 Merge branch 'master' of git.phyks.me:Phyks/flatisfy 2017-11-09 16:01:00 +01:00
Lucas Verney b9e108d63d Make Flat notes searchable 2017-11-09 16:00:50 +01:00
Lucas Verney 2d8dfcb7c7 Fix a bug with minimum number of photos constraint 2017-11-03 17:41:16 +01:00
Lucas Verney 1baebffb87 Do some linting 2017-10-29 21:04:09 +01:00
Lucas Verney 40e62baadf Add a config option to filter on terms in the description. Fix #77. 2017-10-29 20:16:33 +01:00
Lucas Verney f81deb9f13 Rename minimum_photos config option to minimum_nb_photos 2017-10-29 20:15:19 +01:00
Lucas Verney 6b82af6b85 Review of Hatrix42 PR 2017-10-29 03:05:35 +01:00
Lucas Verney 9c55904f59 Merge branch 'master' of https://github.com/Hatrix42/Flatisfy into Hatrix42-master 2017-10-29 02:58:02 +01:00
Lucas Verney 1791142218 Check that area, rooms etc are lists in config 2017-10-29 02:39:15 +02:00
Lucas Verney 1d6233e5ca Completely fix https://github.com/Phyks/Flatisfy/issues/10 2017-10-24 16:56:23 -04:00
Lucas Verney 7fd44b7e8f No longer make queries with more than three cities 
Some postal codes correspond to multiple cities, then sometimes we were
spawning queries with more than three cities. This should no longer be
the case. This fixes https://github.com/Phyks/Flatisfy/issues/10.
 2017-10-24 15:50:19 -04:00
Maël Le Garrec$ e7c81d31ac Add config option for minimum number of pictures 2017-10-24 17:21:48 +02:00
Lucas Verney c96b2a3f4c Fix for https://github.com/Phyks/Flatisfy/issues/8 2017-10-19 15:34:02 -04:00
Lucas Verney f1a5535bca Fix an error introduced in 88a923c87e 2017-10-19 15:30:03 -04:00
Lucas Verney fa20d0fcbc Fix #73 2017-09-27 00:16:36 +02:00
Lucas Verney 9b087729d3 Fix flatisfy.tools.batch with Python3 2017-09-26 01:04:09 +02:00
Lucas Verney 7e35b71c28 Fix an issue with functools, see https://github.com/Phyks/Flatisfy/issues/4 2017-09-24 23:10:07 +02:00
Lucas Verney 88a923c87e Fix for https://github.com/Phyks/Flatisfy/issues/5 2017-09-24 22:51:12 +02:00
Gautier Pelloux-Prayer 28d4b1e36b Add missing brackets for method invokation 2017-09-24 18:54:46 +02:00
Lucas Verney 01e9f7ec0a Merge branch 'cost-per-sq-meter' into 'master' 
Display cost per square meter

See merge request !8
 2017-07-11 15:08:41 +02:00
Lucas Verney 215fe14d01 Better email notifications 
Rework a bit on top of @bnjbvr's MR to add basic email notifications
support.

* Cleaned the code a bit.
* Add a `Date` and a `Message-ID` header to improve the spam score.

Thanks a lot @bnjbvr's for the MR!
 2017-07-06 15:06:13 +02:00
Benjamin Bouvier 23bbee8271 Adds basic email notifications when new flats have been found (fixes #47). 2017-07-06 15:06:13 +02:00
Lucas Verney 4c07fc8ba1 Merge branch 'better-dedup' into 'master' 
Optimize photo comparisons

Closes #41 and #59

See merge request !9
 2017-06-29 12:04:44 +02:00
Lucas Verney be43206d4b Merge branch 'fix-slider' into 'master' 
Fix image slider perspective

See merge request !3
 2017-06-29 12:00:51 +02:00
Benjamin Bouvier 6a0681fc99 Optimize photo comparisons; fixes #41, fixes #59 2017-06-28 22:37:13 +02:00
Benjamin Bouvier 26e2cdefbe Display cost per square meter; 2017-06-28 22:03:20 +02:00
Lucas Verney 210192d09c Rework a bit on top of bnjbvr's fix for #49. 
* Remove the text-decoration due to the use of links
* Make the link element occupy the whole table cell, without padding
* Use `router-link` instead of `a` tag
 2017-06-26 16:26:35 +02:00
Benjamin Bouvier aac6b63e0a Use simple anchors for flat details; 2017-06-26 15:52:51 +02:00
Lucas Verney 6ed484bb8b Merge branch 'foncia' into 'master' 
Add support for Foncia backend from Weboob

See merge request !5
 2017-06-26 13:33:15 +02:00
Lucas Verney 5c171a8356 Do not constrain too much the minimum zoom level of the flats map. 2017-06-20 16:32:27 +02:00
Lucas Verney 78b9ebf04e Better handling of phone links 
Some Weboob backends were returning multiple phone numbers
(comma-separated) or phone numbers in different formats (with or without
spaces, with or without separating dots, etc).

This commit harmonizes all of this so that all phone links will be
rendered correctly and be usable to call in a single click.

Related, https://git.weboob.org/weboob/devel/merge_requests/42.
 2017-06-20 14:20:54 +02:00
Lucas Verney f1df23dc29 Linting 2017-06-20 13:37:54 +02:00
Lucas Verney 054c5679bb Handle exceptions raised when fetching details 
Weboob does handle the raised exceptions when fetching housing posts
through the `WebNip` class. However, when fetching details, we directly
call the Weboob backend method and no exceptions handling is done.

When importing data with a `crontask`, websites may be offline for some
time, especially at night, then leading to a failure in import process.

Let's just handle such exceptions so that an offline website does not
break the whole import process. Details from the temporary offline
websites which could not be fetched will be fetched at next run.

This closes #60.
 2017-06-20 13:37:54 +02:00
Lucas Verney 6e12490fbd Add some doc about CLI option and better informative message 2017-06-19 17:20:53 +02:00
Lucas Verney 082d177f19 More informative error message when matching accidentally a wrong postal code. See #64 for more context. 2017-06-19 17:15:42 +02:00
Lucas Verney ed7e9dfc1a Add a CLI argument to import only a given constraint. Drop the necessary constraint. 2017-06-19 16:08:06 +02:00
Lucas Verney b7f9427d53 Fix GPS position import of Tcl stations 
Tcl stations GPS coordinates are given as `lng, lat` and not `lat, lng`
hence it was not working because the found stations were too far from
the housing post location, as reported by @bnjbvr.

This commit should fix this.
 2017-06-19 14:25:02 +02:00
Lucas Verney c4f6a80b0c Fix a StopIteration error in web API 2017-06-19 12:01:55 +02:00
Lucas Verney b187a106e4 Avoid loading multiple times the same data 
There was a bug in `data.py` which made it load the same data as many
times as the number of postal codes in constraints. This is now fixed.
 2017-06-19 12:01:10 +02:00
Lucas Verney d396d33506 Rewrite doc and frontend to match new constraints in config 2017-06-16 16:56:59 +02:00
Lucas Verney bdf8a6b8d2 Handle multiple constraints in the config 
All the backend part has been rewritten to handle multiple constraints
in the config (== multiple queries).

Also did some linting.

Still to be done: frontend part and doc.
 2017-06-16 16:22:02 +02:00
Lucas Verney 12a55e64be Add support for Tcl data 
This closes #58.
 2017-06-15 15:52:11 +02:00
Lucas Verney 82a88590b6 Rewrite data files management, to make it more modular 
This is to prepare for the integration of Tcl data, see #58.
 2017-06-15 15:48:16 +02:00
Lucas Verney 947f53252a Explain how to enable new Weboob backends in Flatisfy in CONTRIBUTING.md 2017-06-14 16:35:36 +02:00
Lucas Verney 1517d8d30b Add support for Foncia backend from Weboob 2017-06-14 16:30:19 +02:00
Lucas Verney 3469035f4a Towards more modular system for data files 
Also use `lru_cache` to do some memoization on data files loading
function, and speed up everything a bit.
 2017-06-14 15:29:33 +02:00
Benjamin Bouvier 4beee77a02 Fixes #55: Handle ValueError when fetching detailed info about a flat; 2017-06-14 11:02:49 +02:00
Lucas Verney 969c6ab0fe There are up to 3 passes available, the `--passes` argument constraint should match. 2017-06-14 10:50:02 +02:00
Lucas Verney f52b07b869 No need for unicode 
Missed it while reviewing !2, but as we use `from builtins import str`, there is no need to use `unicode`.
 2017-06-14 10:48:17 +02:00
Benjamin Bouvier 8c126ef8e3 Catch the right error when the constraint type name is not set; 2017-06-13 16:17:03 +02:00
Lucas Verney adb520b54d SHow "Loading…" message when loading data from the API 2017-05-12 16:27:31 +02:00
Lucas Verney e4768de086 Fix a bug introduced in last commit, do not pre-wrap TD cells. 2017-05-12 11:46:37 +02:00
Lucas Verney 03e6571545 Fix broken table layout due to large notes in pre tag 
This closes #51.
 2017-05-11 14:27:30 +02:00
Lucas Verney 2bdfa8158b Fix sorting order in Chromium 
This closes #48.
 2017-05-11 14:06:38 +02:00
Lucas Verney 6272819109 A click on a button in a flat row in the flats table should not trigger 
a click event and go to the flat details page.
 2017-05-09 17:50:07 +02:00
Lucas Verney d152b4dce8 Better "Followed" page 
"Followed" page is now sorted by notation for each postal code. Notes
left about the flats are listed in the list, to help have a good
overview of the followed flats.

This closes issue #45.
 2017-05-09 17:36:17 +02:00
Lucas Verney 05aa5812ad Better mobile UI 
The mobile UI is now more responsive, without grid in the details view.
Clicking on a line in the flats table sends the user to the flat
details. Also features better title descriptions and reactive title of
the tab.
 2017-05-09 16:02:24 +02:00
Lucas Verney 7cec9dcec2 Prefill search input from query 2017-05-05 11:12:57 +02:00
Lucas Verney e0f8434877 Fix whooshalchemy 
Whooshalchemy was adding an event every time a new db session was
created, thus increasing a lot the computation time at each new call.

This is now fixed, and is a partial fix for #43.
 2017-05-05 11:04:29 +02:00
Lucas Verney a5aeadef86
Various bug fixes 
Fix some bugs introduced in recent developments as well as some UI
issues (and i18n).

Closes issue #42.
 2017-05-04 20:52:10 +02:00
Lucas Verney 49f5b6a714 Status of flats was not correctly set at first import 
When importing flats for the first time, their status was not correctly
set, resulting in flats not being marked as ignored when they should be.

This is a bug introduced with the fix for issue #31, now fixed.
 2017-05-04 15:33:20 +02:00
Lucas Verney 0e3d1576b2
Use stars to note flats 
Instead of a binary "followed" / "not followed" status, use 5 stars to
allow users to give a note to a flat between 0 (not followed) and 5. Any
note different from zero add a "followed" status.

Closes issue #36.
 2017-05-03 22:14:51 +02:00
Lucas Verney 69588a9601
Add the possibility to leave notes on flats 
* Add a database field to store `notes` (as a memo) about flats.
* Add matching UI elements to let users store their notes about flats.

This commit closes issue #34.
 2017-05-03 19:18:07 +02:00
Lucas Verney 8a50dd3302 Do not refetch details on filtering. To fetch details, one should do import. Closes issue #38. 2017-05-03 15:55:08 +02:00
Lucas Verney 982ea995a7 Add a search feature and matching UI elements 
Also do some minor UI improvements:
* Add an icon to identify followed flats in tables
* Fix wrong enforced plural for 'flats'
 2017-05-03 15:55:08 +02:00
Lucas Verney 3df3162e2a Be careful when filtering / reimporting, do not overwrite user-defined status. Closes #31. 2017-05-03 15:55:07 +02:00
Lucas Verney b0e18b45b7 Fix bugs in Chromium. Closes issue #33. 2017-05-03 15:55:07 +02:00
Lucas Verney aae71282aa Better display of time_to, close issue #32. 2017-05-03 15:55:07 +02:00
Lucas Verney 589bfdfb13 Better deduplication 
Perform deeper deduplication, based on all the available data, and
trying to match common photos.
 2017-05-03 15:55:07 +02:00
Lucas Verney 2af742b764 Various UI fixes 
* Fix status on details page
* Fix follow button on details page
* Replace back button by a link on the flat status
* Fix leaflet markers position, close #24
* Fix Leaflet icons
* Add unfollow link on followed posts
 2017-05-03 15:55:07 +02:00
Lucas Verney bd3e599d12 Handle weboob exceptions, fix issue #25 2017-05-03 15:55:07 +02:00
Lucas Verney 0fb5f28184 Add a page listing flats by status 
Also display the journeys on the details view.
 2017-05-03 15:55:07 +02:00
Lucas Verney 5f2f4d0ccf Better deduplication 
* Improve deduplication on URLs (match sets).
* Keep track of duplicates and update their status on refiltering.
 2017-05-03 15:55:07 +02:00
Lucas Verney 1d98c631e0 Refilter command and backends in config 
* Add a refilter command
* Add a backend option in config to only enable some backends.
 2017-05-03 15:55:07 +02:00
Lucas Verney 65bd5db98a Fix Decimal issue with JSON 2017-05-03 15:55:06 +02:00
Lucas Verney 847a5954d5 Send credentials with API calls, to make HTTP auth work 
Also serve OSM tiles with HTTPS
 2017-05-03 15:55:06 +02:00
Lucas Verney f7c7459d96 Add screen reader description and a favicon 
Closes issue #20.
 2017-05-03 15:55:06 +02:00
Lucas Verney 3635e7c559 FlatsTable columns should be sortable 
Closes issue #4.
 2017-05-03 15:55:06 +02:00
Lucas Verney 533638e16d Back did not work on flat details page sometimes 
Closes issue #13.
 2017-05-03 15:55:06 +02:00
Lucas Verney 5f47b0ff65 Write some documentation 2017-05-03 15:54:59 +02:00
Lucas Verney a57d9ce8e3 Switch to a Vue-based web app 
* Init Webpack / Babel / etc setup.
* Build the app using Vue, Vue-router, Vuex.
* i18n

Some backends changes were made to match the webapp development:
* Return the flat status as a single string ("new" rather than
"FlatStatus.new")

* Completely switch to calling Weboob API directly for fetching
* Use Canister for Bottle logging
* Handle merging of details dict better
* Add a WSGI script
* Keep track of duplicates
* Webserver had to be restarted to fetch external changes to the db
* Handle leboncoin module better

Also add contributions guidelines.

Closes issue #3
Closes issue #14.
 2017-05-03 15:54:26 +02:00
Lucas Verney 4966fe2111
Styling update, fix some Pylint errors 
Also update the stations fetching code to handle the warn the user when
fetched stations differ from the one fetched at previous pass.
 2017-04-13 23:22:11 +02:00
Lucas Verney d7012e3834
Initial commit 2017-04-13 20:03:59 +02:00