From 1168273efa0f941eae734645dc2d3d82ae778016 Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Mon, 17 Sep 2018 20:48:23 -0500 Subject: [PATCH 01/31] FutureWarning from groupby. Warning about elementwise comparison failing when we indexed a dataframe with boolean dataframe --- doc/source/cookbook.rst | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/doc/source/cookbook.rst b/doc/source/cookbook.rst index f6fa9e9f86143..a4dc99383a562 100644 --- a/doc/source/cookbook.rst +++ b/doc/source/cookbook.rst @@ -505,13 +505,11 @@ Unlike agg, apply's callable is passed a sub-DataFrame which gives you access to .. ipython:: python df = pd.DataFrame({'A' : [1, 1, 2, 2], 'B' : [1, -1, 1, 2]}) - gb = df.groupby('A') def replace(g): - mask = g < 0 - g.loc[mask] = g[~mask].mean() - return g + mask = g < 0 + return g.where(mask, g[~mask].mean()) gb.transform(replace) From 41c8297142bc95522c199a0f623abcf77d45a8a5 Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Mon, 17 Sep 2018 20:48:57 -0500 Subject: [PATCH 02/31] Purge read_table --- doc/source/io.rst | 29 +++++++++++++---------------- 1 file changed, 13 insertions(+), 16 deletions(-) diff --git a/doc/source/io.rst b/doc/source/io.rst index c2c8c1c17700f..84530b2b560d0 100644 --- a/doc/source/io.rst +++ b/doc/source/io.rst @@ -66,16 +66,13 @@ The pandas I/O API is a set of top level ``reader`` functions accessed like CSV & Text files ---------------- -The two workhorse functions for reading text files (a.k.a. flat files) are -:func:`read_csv` and :func:`read_table`. They both use the same parsing code to -intelligently convert tabular data into a ``DataFrame`` object. See the -:ref:`cookbook` for some advanced strategies. +The workhorse function for reading text files (a.k.a. flat files) is +:func:`read_csv`. See the :ref:`cookbook` for some advanced strategies. Parsing options ''''''''''''''' -The functions :func:`read_csv` and :func:`read_table` accept the following -common arguments: +:func:`read_csv` accepts the following common arguments: Basic +++++ @@ -780,8 +777,8 @@ Date Handling Specifying Date Columns +++++++++++++++++++++++ -To better facilitate working with datetime data, :func:`read_csv` and -:func:`read_table` use the keyword arguments ``parse_dates`` and ``date_parser`` +To better facilitate working with datetime data, :func:`read_csv` +uses the keyword arguments ``parse_dates`` and ``date_parser`` to allow users to specify a variety of columns and date/time formats to turn the input text data into ``datetime`` objects. @@ -1434,7 +1431,7 @@ Suppose you have data indexed by two columns: print(open('data/mindex_ex.csv').read()) -The ``index_col`` argument to ``read_csv`` and ``read_table`` can take a list of +The ``index_col`` argument to ``read_csv`` can take a list of column numbers to turn multiple columns into a ``MultiIndex`` for the index of the returned object: @@ -1505,8 +1502,8 @@ class of the csv module. For this, you have to specify ``sep=None``. .. ipython:: python - print(open('tmp2.sv').read()) - pd.read_csv('tmp2.sv', sep=None, engine='python') + print(open('tmp2.sv').read()) + pd.read_csv('tmp2.sv', sep=None, engine='python') .. _io.multiple_files: @@ -1528,16 +1525,16 @@ rather than reading the entire file into memory, such as the following: .. ipython:: python print(open('tmp.sv').read()) - table = pd.read_table('tmp.sv', sep='|') + table = pd.read_csv('tmp.sv', sep='|') table -By specifying a ``chunksize`` to ``read_csv`` or ``read_table``, the return +By specifying a ``chunksize`` to ``read_csv``, the return value will be an iterable object of type ``TextFileReader``: .. ipython:: python - reader = pd.read_table('tmp.sv', sep='|', chunksize=4) + reader = pd.read_csv('tmp.sv', sep='|', chunksize=4) reader for chunk in reader: @@ -1548,7 +1545,7 @@ Specifying ``iterator=True`` will also return the ``TextFileReader`` object: .. ipython:: python - reader = pd.read_table('tmp.sv', sep='|', iterator=True) + reader = pd.read_csv('tmp.sv', sep='|', iterator=True) reader.get_chunk(5) .. ipython:: python @@ -3067,7 +3064,7 @@ Clipboard A handy way to grab data is to use the :meth:`~DataFrame.read_clipboard` method, which takes the contents of the clipboard buffer and passes them to the -``read_table`` method. For instance, you can copy the following text to the +``read_csv`` method. For instance, you can copy the following text to the clipboard (CTRL-C on many operating systems): .. code-block:: python From 2e76e845b23f036d96b80fa164d49b577481303e Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Mon, 17 Sep 2018 20:49:26 -0500 Subject: [PATCH 03/31] Removed nested list example --- doc/source/text.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/doc/source/text.rst b/doc/source/text.rst index 61583a179e572..d01c48695d0d6 100644 --- a/doc/source/text.rst +++ b/doc/source/text.rst @@ -312,14 +312,15 @@ All one-dimensional list-likes can be combined in a list-like container (includi s u - s.str.cat([u.values, ['A', 'B', 'C', 'D'], map(str, u.index)], na_rep='-') + s.str.cat([u.values, + u.index.astype(str).values], na_rep='-') All elements must match in length to the calling ``Series`` (or ``Index``), except those having an index if ``join`` is not None: .. ipython:: python v - s.str.cat([u, v, ['A', 'B', 'C', 'D']], join='outer', na_rep='-') + s.str.cat([u, v], join='outer', na_rep='-') If using ``join='right'`` on a list of ``others`` that contains different indexes, the union of these indexes will be used as the basis for the final concatenation: From a70f86df6b57b67cf5c7035f1f0c4afd7642da31 Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Mon, 17 Sep 2018 21:45:27 -0500 Subject: [PATCH 04/31] Fixed resample __iter__ --- doc/source/timeseries.rst | 41 +++++++++++++++++++++++---------------- 1 file changed, 24 insertions(+), 17 deletions(-) diff --git a/doc/source/timeseries.rst b/doc/source/timeseries.rst index 71bc064ffb0c2..b2b324c14b318 100644 --- a/doc/source/timeseries.rst +++ b/doc/source/timeseries.rst @@ -758,11 +758,21 @@ natural and functions similarly to :py:func:`itertools.groupby`: .. ipython:: python - resampled = df.resample('H') + small = pd.Series( + range(6), + index=pd.to_datetime(['2017-01-01T00:00:00', + '2017-01-01T00:30:00', + '2017-01-01T00:31:00', + '2017-01-01T01:00:00', + '2017-01-01T03:00:00', + '2017-01-01T03:05:00']) + ) + resampled = small.resample('H') for name, group in resampled: - print(name) - print(group) + print("Group: ", name) + print("-" * 27) + print(group, end="\n\n") See :ref:`groupby.iterating-label`. @@ -910,26 +920,22 @@ It's definitely worth exploring the ``pandas.tseries.offsets`` module and the various docstrings for the classes. These operations (``apply``, ``rollforward`` and ``rollback``) preserve time -(hour, minute, etc) information by default. To reset time, use ``normalize=True`` -when creating the offset instance. If ``normalize=True``, the result is -normalized after the function is applied. - +(hour, minute, etc) information by default. To reset time, use ``normalize`` +before or after applying the operation (depending on whether you want the +time information included in the operation. .. ipython:: python + ts = pd.Timestamp('2014-01-01 09:00') day = Day() - day.apply(pd.Timestamp('2014-01-01 09:00')) - - day = Day(normalize=True) - day.apply(pd.Timestamp('2014-01-01 09:00')) + day.apply(ts) + day.apply(ts).normalize() + ts = pd.Timestamp('2014-01-01 22:00') hour = Hour() - hour.apply(pd.Timestamp('2014-01-01 22:00')) - - hour = Hour(normalize=True) - hour.apply(pd.Timestamp('2014-01-01 22:00')) - hour.apply(pd.Timestamp('2014-01-01 23:00')) - + hour.apply(ts) + hour.apply(ts).normalize() + hour.apply(pd.Timestamp("2014-01-01 23:30")).normalize() .. _timeseries.dayvscalendarday: @@ -1488,6 +1494,7 @@ time. The method for this is :meth:`~Series.shift`, which is available on all of the pandas objects. .. ipython:: python + ts = pd.Series(range(len(rng)), index=rng) ts = ts[:5] ts.shift(1) From e4a8b064b9c93378430ebcf972e999349f84f05b Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Mon, 17 Sep 2018 21:52:51 -0500 Subject: [PATCH 05/31] Old whatsnew --- doc/source/whatsnew/v0.18.0.txt | 2 +- doc/source/whatsnew/v0.20.0.txt | 2 +- doc/source/whatsnew/v0.24.0.txt | 5 +++-- 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/doc/source/whatsnew/v0.18.0.txt b/doc/source/whatsnew/v0.18.0.txt index a3213136d998a..e38ba54d4b058 100644 --- a/doc/source/whatsnew/v0.18.0.txt +++ b/doc/source/whatsnew/v0.18.0.txt @@ -373,7 +373,7 @@ New Behavior: s = pd.Series([1,2,3], index=np.arange(3.)) s s.index - print(s.to_csv(path=None)) + print(s.to_csv(path_or_buf=None, header=False)) Changes to dtype assignment behaviors ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/doc/source/whatsnew/v0.20.0.txt b/doc/source/whatsnew/v0.20.0.txt index 3c0818343208a..819f24254b2ce 100644 --- a/doc/source/whatsnew/v0.20.0.txt +++ b/doc/source/whatsnew/v0.20.0.txt @@ -186,7 +186,7 @@ Previously, only ``gzip`` compression was supported. By default, compression of URLs and paths are now inferred using their file extensions. Additionally, support for bz2 compression in the python 2 C-engine improved (:issue:`14874`). -.. ipython:: python +.. code-block:::: python url = 'https://github.com/{repo}/raw/{branch}/{path}'.format( repo = 'pandas-dev/pandas', diff --git a/doc/source/whatsnew/v0.24.0.txt b/doc/source/whatsnew/v0.24.0.txt index 3a44b0260153c..05f428dc3d501 100644 --- a/doc/source/whatsnew/v0.24.0.txt +++ b/doc/source/whatsnew/v0.24.0.txt @@ -461,9 +461,10 @@ all-``NaT``. This is for compatibility with ``TimedeltaIndex`` and ``Series`` behavior (:issue:`22163`) .. ipython:: python + :okexcept: - df = pd.DataFrame([pd.Timedelta(days=1)]) - df - np.nan + df = pd.DataFrame([pd.Timedelta(days=1)]) + df - np.nan Previous Behavior: From 693eead60d96d4297d9b55a6e86eeeb2b2797220 Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Mon, 17 Sep 2018 21:52:59 -0500 Subject: [PATCH 06/31] Ecosystem --- doc/source/ecosystem.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/source/ecosystem.rst b/doc/source/ecosystem.rst index 1014982fea21a..7fffcadd8ee8c 100644 --- a/doc/source/ecosystem.rst +++ b/doc/source/ecosystem.rst @@ -73,8 +73,8 @@ large data to thin clients. `seaborn `__ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Seaborn is a Python visualization library based on `matplotlib -`__. It provides a high-level, dataset-oriented +Seaborn is a Python visualization library based on +`matplotlib `__. It provides a high-level, dataset-oriented interface for creating attractive statistical graphics. The plotting functions in seaborn understand pandas objects and leverage pandas grouping operations internally to support concise specification of complex visualizations. Seaborn @@ -140,7 +140,7 @@ which are utilized by Jupyter Notebook for displaying (Note: HTML tables may or may not be compatible with non-HTML Jupyter output formats.) -See :ref:`Options and Settings ` and :ref:`` +See :ref:`Options and Settings ` and :ref:`options.available ` for pandas ``display.`` settings. `quantopian/qgrid `__ @@ -169,7 +169,7 @@ or the clipboard into a new pandas DataFrame via a sophisticated import wizard. Most pandas classes, methods and data attributes can be autocompleted in Spyder's `Editor `__ and `IPython Console `__, -and Spyder's `Help pane`__ can retrieve +and Spyder's `Help pane `__ can retrieve and render Numpydoc documentation on pandas objects in rich text with Sphinx both automatically and on-demand. From e6b2c09284f2d68e38c1d5df06a966817a802d63 Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Mon, 17 Sep 2018 22:06:26 -0500 Subject: [PATCH 07/31] to_csv --- pandas/core/generic.py | 27 +++++++++++++++++++-------- 1 file changed, 19 insertions(+), 8 deletions(-) diff --git a/pandas/core/generic.py b/pandas/core/generic.py index 373830ec7892e..e1c6ad978c32e 100644 --- a/pandas/core/generic.py +++ b/pandas/core/generic.py @@ -9513,8 +9513,11 @@ def to_csv(self, path_or_buf=None, sep=",", na_rep='', float_format=None, path_or_buf : string or file handle, default None File path or object, if None is provided the result is returned as a string. + .. versionchanged:: 0.24.0 - Was previously named "path" for Series. + + Was previously named "path" for Series. + sep : character, default ',' Field delimiter for the output file. na_rep : string, default '' @@ -9526,8 +9529,11 @@ def to_csv(self, path_or_buf=None, sep=",", na_rep='', float_format=None, header : boolean or list of string, default True Write out the column names. If a list of strings is given it is assumed to be aliases for the column names + .. versionchanged:: 0.24.0 - Previously defaulted to False for Series. + + Previously defaulted to False for Series. + index : boolean, default True Write row names (index) index_label : string or sequence, or False, default None @@ -9541,14 +9547,16 @@ def to_csv(self, path_or_buf=None, sep=",", na_rep='', float_format=None, encoding : string, optional A string representing the encoding to use in the output file, defaults to 'ascii' on Python 2 and 'utf-8' on Python 3. - compression : {'infer', 'gzip', 'bz2', 'zip', 'xz', None}, - default 'infer' - If 'infer' and `path_or_buf` is path-like, then detect compression - from the following extensions: '.gz', '.bz2', '.zip' or '.xz' - (otherwise no compression). + compression : {'infer', 'gzip', 'bz2', 'zip', 'xz', None} + + If 'infer' (the defualt) and `path_or_buf` is path-like, then + detect compression from the following extensions: '.gz', '.bz2', + '.zip' or '.xz' (otherwise no compression). .. versionchanged:: 0.24.0 + 'infer' option added and set to default + line_terminator : string, default ``'\n'`` The newline character or character sequence to use in the output file @@ -9565,7 +9573,9 @@ def to_csv(self, path_or_buf=None, sep=",", na_rep='', float_format=None, chunksize : int or None rows to write at a time tupleize_cols : boolean, default False + .. deprecated:: 0.21.0 + This argument will be removed and will always write each row of the multi-index as a separate row in the CSV file. @@ -9579,7 +9589,8 @@ def to_csv(self, path_or_buf=None, sep=",", na_rep='', float_format=None, European data .. versionchanged:: 0.24.0 - The order of arguments for Series was changed. + + The order of arguments for Series was changed. """ df = self if isinstance(self, ABCDataFrame) else self.to_frame() From a7f0b3841367afc5ed641287e1bb971243477d0f Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Mon, 17 Sep 2018 22:07:59 -0500 Subject: [PATCH 08/31] to_json --- pandas/core/generic.py | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/pandas/core/generic.py b/pandas/core/generic.py index e1c6ad978c32e..0b4053467408e 100644 --- a/pandas/core/generic.py +++ b/pandas/core/generic.py @@ -2061,9 +2061,10 @@ def to_json(self, path_or_buf=None, orient=None, date_format=None, .. versionadded:: 0.19.0 compression : {'infer', 'gzip', 'bz2', 'zip', 'xz', None}, - default 'infer' + A string representing the compression to use in the output file, - only used when the first argument is a filename. + only used when the first argument is a filename. By default, the + compression is inferred from the filename. .. versionadded:: 0.21.0 .. versionchanged:: 0.24.0 From ff3d2dd7620eb9b4cf928dd4b66dadc7bb65b011 Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Tue, 18 Sep 2018 10:29:39 -0500 Subject: [PATCH 09/31] Handle subpackages better --- doc/make.py | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/doc/make.py b/doc/make.py index d85747458148d..89766f89cda75 100755 --- a/doc/make.py +++ b/doc/make.py @@ -135,6 +135,12 @@ def _process_single_doc(self, single_doc): try: obj = pandas # noqa: F821 for name in single_doc.split('.'): + try: + # for names not in the top-level namespace by default, + # e.g. pandas.io.formats.style.Styler + importlib.import_module('.'.join([obj.__name__, name])) + except ModuleNotFoundError: + pass obj = getattr(obj, name) except AttributeError: raise ValueError('Single document not understood, it should ' From e54424981814f2ec5a5047e3ff33953340058b3a Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Tue, 18 Sep 2018 10:30:04 -0500 Subject: [PATCH 10/31] Fixed unexpected indent --- pandas/io/formats/style.py | 1 + 1 file changed, 1 insertion(+) diff --git a/pandas/io/formats/style.py b/pandas/io/formats/style.py index b175dd540a518..f4bb53ba4f218 100644 --- a/pandas/io/formats/style.py +++ b/pandas/io/formats/style.py @@ -1073,6 +1073,7 @@ def bar(self, subset=None, axis=0, color='#d65f5f', width=100, percent of the cell's width. align : {'left', 'zero',' mid'}, default 'left' How to align the bars with the cells. + - 'left' : the min value starts at the left of the cell. - 'zero' : a value of zero is located at the center of the cell. - 'mid' : the center of the cell is at (max-min)/2, or From 8bdb9202e95264f6e1009e8afb66c071c03b9f76 Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Tue, 18 Sep 2018 10:33:31 -0500 Subject: [PATCH 11/31] Fixed "inline interpreted text..." --- pandas/core/window.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/pandas/core/window.py b/pandas/core/window.py index eed0e97f30dc9..30073e363846a 100644 --- a/pandas/core/window.py +++ b/pandas/core/window.py @@ -1404,7 +1404,7 @@ def _get_cov(X, Y): otherwise defaults to `False`. Not relevant for :class:`~pandas.Series`. **kwargs - Under Review. + Unused. Returns ------- @@ -1430,7 +1430,7 @@ def _get_cov(X, Y): all 1's), except for :class:`~pandas.DataFrame` inputs with `pairwise` set to `True`. - Function will return `NaN`s for correlations of equal valued sequences; + Function will return ``NaN`` for correlations of equal valued sequences; this is the result of a 0/0 division error. When `pairwise` is set to `False`, only matching columns between `self` and @@ -1446,7 +1446,7 @@ def _get_cov(X, Y): Examples -------- The below example shows a rolling calculation with a window size of - four matching the equivalent function call using `numpy.corrcoef`. + four matching the equivalent function call using :meth:`numpy.corrcoef`. >>> v1 = [3, 3, 3, 5, 8] >>> v2 = [3, 4, 4, 4, 8] From ae0f8ff028076ece045736a847ced25ddc5f5086 Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Tue, 18 Sep 2018 10:36:20 -0500 Subject: [PATCH 12/31] Fixed "malformed hyperlink target" --- doc/source/whatsnew/v0.24.0.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/source/whatsnew/v0.24.0.txt b/doc/source/whatsnew/v0.24.0.txt index 05f428dc3d501..77ce495a5cbc7 100644 --- a/doc/source/whatsnew/v0.24.0.txt +++ b/doc/source/whatsnew/v0.24.0.txt @@ -450,7 +450,7 @@ Previous Behavior: Out[3]: Int64Index([0, 1, 2], dtype='int64') -.. _whatsnew_0240.api.timedelta64_subtract_nan +.. _whatsnew_0240.api.timedelta64_subtract_nan: Addition/Subtraction of ``NaN`` from :class:`DataFrame` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ From a275dfb4ead2382346b862b91a483b47230d1dc6 Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Tue, 18 Sep 2018 10:55:25 -0500 Subject: [PATCH 13/31] Add warnings to CLI --- doc/make.py | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/doc/make.py b/doc/make.py index 89766f89cda75..fa5780a29a24a 100755 --- a/doc/make.py +++ b/doc/make.py @@ -78,7 +78,7 @@ class DocBuilder: script. """ def __init__(self, num_jobs=1, include_api=True, single_doc=None, - verbosity=0): + verbosity=0, allow_warnings=False): self.num_jobs = num_jobs self.include_api = include_api self.verbosity = verbosity @@ -87,6 +87,7 @@ def __init__(self, num_jobs=1, include_api=True, single_doc=None, if single_doc is not None: self._process_single_doc(single_doc) self.exclude_patterns = self._exclude_patterns + self.allow_warnings = allow_warnings self._generate_index() if self.single_doc_type == 'docstring': @@ -233,10 +234,10 @@ def _sphinx_build(self, kind): if kind not in ('html', 'latex', 'spelling'): raise ValueError('kind must be html, latex or ' 'spelling, not {}'.format(kind)) - self._run_os('sphinx-build', '-j{}'.format(self.num_jobs), '-b{}'.format(kind), + '{}'.format("" if self.allow_warnings else "-W"), '-{}'.format( 'v' * self.verbosity) if self.verbosity else '', '-d{}'.format(os.path.join(BUILD_PATH, 'doctrees')), @@ -355,6 +356,10 @@ def main(): argparser.add_argument('-v', action='count', dest='verbosity', default=0, help=('increase verbosity (can be repeated), ' 'passed to the sphinx build command')) + argparser.add_argument("--allow-warnings", + default=False, + action="store_true", + help="Whether to allow warnings in the build.") args = argparser.parse_args() if args.command not in cmds: @@ -374,7 +379,7 @@ def main(): os.environ['MPLBACKEND'] = 'module://matplotlib.backends.backend_agg' builder = DocBuilder(args.num_jobs, not args.no_api, args.single, - args.verbosity) + args.verbosity, args.allow_warnings) getattr(builder, args.command)() From b1908660804fe897ebf38c71aa85e1811bc1e73a Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Tue, 18 Sep 2018 10:56:49 -0500 Subject: [PATCH 14/31] Fixed unexpected indentation --- pandas/core/series.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/pandas/core/series.py b/pandas/core/series.py index 8f69de973e7a3..5b0b75cfcbce2 100644 --- a/pandas/core/series.py +++ b/pandas/core/series.py @@ -2746,6 +2746,7 @@ def nlargest(self, n=5, keep='first'): keep : {'first', 'last', 'all'}, default 'first' When there are duplicate values that cannot all fit in a Series of `n` elements: + - ``first`` : take the first occurrences based on the index order - ``last`` : take the last occurrences based on the index order - ``all`` : keep all occurrences. This can result in a Series of @@ -2841,6 +2842,7 @@ def nsmallest(self, n=5, keep='first'): keep : {'first', 'last', 'all'}, default 'first' When there are duplicate values that cannot all fit in a Series of `n` elements: + - ``first`` : take the first occurrences based on the index order - ``last`` : take the last occurrences based on the index order - ``all`` : keep all occurrences. This can result in a Series of From a46e4c7b513e7930e6a5e1b6e530ce223f9613a6 Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Tue, 18 Sep 2018 10:59:16 -0500 Subject: [PATCH 15/31] newline after directive --- doc/source/whatsnew/v0.24.0.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/source/whatsnew/v0.24.0.txt b/doc/source/whatsnew/v0.24.0.txt index 77ce495a5cbc7..34017d582602b 100644 --- a/doc/source/whatsnew/v0.24.0.txt +++ b/doc/source/whatsnew/v0.24.0.txt @@ -246,7 +246,6 @@ UTC offset (:issue:`17697`, :issue:`11736`, :issue:`22457`) .. code-block:: ipython - In [2]: pd.to_datetime("2015-11-18 15:30:00+05:30") Out[2]: Timestamp('2015-11-18 10:00:00') @@ -284,6 +283,7 @@ Passing ``utc=True`` will mimic the previous behavior but will correctly indicat that the dates have been converted to UTC .. ipython:: python + pd.to_datetime(["2015-11-18 15:30:00+05:30", "2015-11-18 16:30:00+06:30"], utc=True) .. _whatsnew_0240.api_breaking.calendarday: From 74af53db4164670dbeafbd58ae45467d15eb8044 Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Tue, 18 Sep 2018 11:00:41 -0500 Subject: [PATCH 16/31] Maybe fix na_value not included in toctree --- doc/source/api.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/source/api.rst b/doc/source/api.rst index e4b055c14ec27..924aaa2ce1762 100644 --- a/doc/source/api.rst +++ b/doc/source/api.rst @@ -2570,6 +2570,7 @@ objects. api.extensions.register_series_accessor api.extensions.register_index_accessor api.extensions.ExtensionDtype + api.extensions.ExtensionDtype.na_value api.extensions.ExtensionArray .. This is to prevent warnings in the doc build. We don't want to encourage From 1668c654422dd5841c32b789a556388d42182192 Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Tue, 18 Sep 2018 11:04:44 -0500 Subject: [PATCH 17/31] Fixed no link to na_value --- doc/source/api.rst | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/doc/source/api.rst b/doc/source/api.rst index 924aaa2ce1762..073ed8a082a11 100644 --- a/doc/source/api.rst +++ b/doc/source/api.rst @@ -2570,7 +2570,6 @@ objects. api.extensions.register_series_accessor api.extensions.register_index_accessor api.extensions.ExtensionDtype - api.extensions.ExtensionDtype.na_value api.extensions.ExtensionArray .. This is to prevent warnings in the doc build. We don't want to encourage @@ -2604,3 +2603,12 @@ objects. generated/pandas.Series.ix generated/pandas.Series.imag generated/pandas.Series.real + + +.. Can't convince sphinx to generate toctree for this class attribute. +.. So we do it manually to avoid a warning + +.. toctree:: + :hidden: + + generated/pandas.api.extensions.ExtensionDtype.na_value From dda2bfc6741bb0d1b07a485cb0910d7e0425c313 Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Tue, 18 Sep 2018 11:04:56 -0500 Subject: [PATCH 18/31] Fixed II ref --- doc/source/basics.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/source/basics.rst b/doc/source/basics.rst index c18b94fea9a28..6eeb97349100a 100644 --- a/doc/source/basics.rst +++ b/doc/source/basics.rst @@ -1935,7 +1935,7 @@ NumPy's type-system for a few cases. * :ref:`Categorical ` * :ref:`Datetime with Timezone ` * :ref:`Period ` -* :ref:`Interval ` +* :ref:`Interval ` Pandas uses the ``object`` dtype for storing strings. From a2b31abada45795987f063e96fde3c638b90c08d Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Tue, 18 Sep 2018 11:07:30 -0500 Subject: [PATCH 19/31] Fixed options ref --- doc/source/ecosystem.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/source/ecosystem.rst b/doc/source/ecosystem.rst index 7fffcadd8ee8c..2e1d573ed4192 100644 --- a/doc/source/ecosystem.rst +++ b/doc/source/ecosystem.rst @@ -140,7 +140,7 @@ which are utilized by Jupyter Notebook for displaying (Note: HTML tables may or may not be compatible with non-HTML Jupyter output formats.) -See :ref:`Options and Settings ` and :ref:`options.available ` +See :ref:`Options and Settings ` and :ref:`options.available` for pandas ``display.`` settings. `quantopian/qgrid `__ From 9f0a948947cf959a8750cbf11b2a52a1ccb47eaf Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Tue, 18 Sep 2018 11:09:31 -0500 Subject: [PATCH 20/31] Fixed link to Resmpaler --- doc/source/timeseries.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/source/timeseries.rst b/doc/source/timeseries.rst index b2b324c14b318..85b0abe421eb2 100644 --- a/doc/source/timeseries.rst +++ b/doc/source/timeseries.rst @@ -753,7 +753,7 @@ regularity will result in a ``DatetimeIndex``, although frequency is lost: Iterating through groups ------------------------ -With the :ref:`Resampler` object in hand, iterating through the grouped data is very +With the ``Resampler`` object in hand, iterating through the grouped data is very natural and functions similarly to :py:func:`itertools.groupby`: .. ipython:: python @@ -774,7 +774,7 @@ natural and functions similarly to :py:func:`itertools.groupby`: print("-" * 27) print(group, end="\n\n") -See :ref:`groupby.iterating-label`. +See :ref:`groupby.iterating-label` or :class:`Resampler.__iter__` for more. .. _timeseries.components: From 158c46dd9b1a143d3ba32e660f48ce90fa9587e9 Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Tue, 18 Sep 2018 16:49:28 -0500 Subject: [PATCH 21/31] Change warning, error, linting --- ci/build_docs.sh | 17 +++++++++++++-- doc/make.py | 54 ++++++++++++++++++++++++++++++++++++++++++------ 2 files changed, 63 insertions(+), 8 deletions(-) diff --git a/ci/build_docs.sh b/ci/build_docs.sh index f445447e3565c..84eb8dec17bae 100755 --- a/ci/build_docs.sh +++ b/ci/build_docs.sh @@ -22,8 +22,21 @@ if [ "$DOC" ]; then echo # Log file for the doc build # echo ############################### - echo ./make.py - ./make.py + echo './make.py 2>&1 | tee doc-build.log' + ./make.py 2>&1 | tee doc-build.log + + echo ################## + echo # Lint build log # + echo ################## + + echo './make.py lint_log --logfile=doc-build.log' + ./make.py lint_log --logfile=doc-build.log + + if [ ?$ == 1 ] + then + echo "Errors in documentation build." + exit 1 + fi echo ######################## echo # Create and send docs # diff --git a/doc/make.py b/doc/make.py index fa5780a29a24a..8460b71846194 100755 --- a/doc/make.py +++ b/doc/make.py @@ -14,6 +14,7 @@ import importlib import sys import os +import textwrap import shutil # import subprocess import argparse @@ -78,7 +79,7 @@ class DocBuilder: script. """ def __init__(self, num_jobs=1, include_api=True, single_doc=None, - verbosity=0, allow_warnings=False): + verbosity=0, warnings_are_errors=False, log_file=None): self.num_jobs = num_jobs self.include_api = include_api self.verbosity = verbosity @@ -87,7 +88,8 @@ def __init__(self, num_jobs=1, include_api=True, single_doc=None, if single_doc is not None: self._process_single_doc(single_doc) self.exclude_patterns = self._exclude_patterns - self.allow_warnings = allow_warnings + self.warnings_are_errors = warnings_are_errors + self.log_file = log_file self._generate_index() if self.single_doc_type == 'docstring': @@ -237,7 +239,7 @@ def _sphinx_build(self, kind): self._run_os('sphinx-build', '-j{}'.format(self.num_jobs), '-b{}'.format(kind), - '{}'.format("" if self.allow_warnings else "-W"), + '{}'.format("W" if self.warnings_are_errors else ""), '-{}'.format( 'v' * self.verbosity) if self.verbosity else '', '-d{}'.format(os.path.join(BUILD_PATH, 'doctrees')), @@ -324,6 +326,42 @@ def spellcheck(self): ' Check pandas/doc/build/spelling/output.txt' ' for more details.') + def lint_log(self): + with open(self.log_file) as f: + log = f.readlines() + + failures = self._check_log(log) + if failures: + self._report_failures(failures) + sys.exit(1) + + @staticmethod + def _check_log(log): + # type: (List[str]) -> List[Tuple[int, str]] + failures = [] + for i, line in enumerate(log): + if "WARNING:" in line: + failures.append((i, line)) + + return failures + + @staticmethod + def _report_failures(failures): + tpl = textwrap.dedent("""\ + {n} failure{s} + + {individual} + """) + joined = [] + for i, (lineno, f) in enumerate(failures): + line = "Failure [{}]: {} (log line {})".format(i, f.strip(), lineno) + joined.append(line) + joined = '\n'.join(joined) + + print(tpl.format(n=len(failures), + s="s" if len(failures) != 1 else "", + individual=joined)) + def main(): cmds = [method for method in dir(DocBuilder) if not method.startswith('_')] @@ -356,10 +394,13 @@ def main(): argparser.add_argument('-v', action='count', dest='verbosity', default=0, help=('increase verbosity (can be repeated), ' 'passed to the sphinx build command')) - argparser.add_argument("--allow-warnings", + argparser.add_argument("--warnings-are-errors", default=False, action="store_true", - help="Whether to allow warnings in the build.") + help="Whether to fail the build on warnings.") + argparser.add_argument("--log-file", + default="doc-build.log", + help="Log file of the build to lint for warnings.") args = argparser.parse_args() if args.command not in cmds: @@ -379,7 +420,8 @@ def main(): os.environ['MPLBACKEND'] = 'module://matplotlib.backends.backend_agg' builder = DocBuilder(args.num_jobs, not args.no_api, args.single, - args.verbosity, args.allow_warnings) + args.verbosity, args.warnings_are_errors, + args.log_file) getattr(builder, args.command)() From c1a5ab802cbea0a2bec9cdee878a0445eac96365 Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Tue, 18 Sep 2018 21:36:38 -0500 Subject: [PATCH 22/31] Sample warning --- doc/source/whatsnew/v0.24.0.txt | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/source/whatsnew/v0.24.0.txt b/doc/source/whatsnew/v0.24.0.txt index 34017d582602b..e55b8e7af8068 100644 --- a/doc/source/whatsnew/v0.24.0.txt +++ b/doc/source/whatsnew/v0.24.0.txt @@ -8,6 +8,8 @@ v0.24.0 (Month XX, 2018) Starting January 1, 2019, pandas feature releases will support Python 3 only. See :ref:`install.dropping-27` for more. +This is a deliberate error :meth:`notaclass` to test the failure. + .. _whatsnew_0240.enhancements: New features From bbcd7bd0c295f38eb1d258dd8eb4af950a558ad7 Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Tue, 18 Sep 2018 21:39:54 -0500 Subject: [PATCH 23/31] update contributing --- doc/.gitignore | 1 + doc/source/contributing.rst | 5 +++++ 2 files changed, 6 insertions(+) create mode 100644 doc/.gitignore diff --git a/doc/.gitignore b/doc/.gitignore new file mode 100644 index 0000000000000..a5dd6ec150da1 --- /dev/null +++ b/doc/.gitignore @@ -0,0 +1 @@ +doc-build.log diff --git a/doc/source/contributing.rst b/doc/source/contributing.rst index 60bfd07961b38..1385fa6cee228 100644 --- a/doc/source/contributing.rst +++ b/doc/source/contributing.rst @@ -345,6 +345,11 @@ Some other important things to know about the docs: Every method should be included in a ``toctree`` in ``api.rst``, else Sphinx will emit a warning. +* The pandas CI system does not allow warnings in the documentation build. + If you cannot discover the cause of the warning from the build output, you can + try elevating warnings to errors with ``python make.py --warnings-are-errors``, + which will immediately halt the build when a warning is encountered. + .. note:: The ``.rst`` files are used to automatically generate Markdown and HTML versions From c8f206c49439bf3cacd90fc379d75201b497a161 Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Tue, 18 Sep 2018 21:49:14 -0500 Subject: [PATCH 24/31] lint --- doc/make.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/doc/make.py b/doc/make.py index 8460b71846194..790c10285a9d5 100755 --- a/doc/make.py +++ b/doc/make.py @@ -349,12 +349,14 @@ def _check_log(log): def _report_failures(failures): tpl = textwrap.dedent("""\ {n} failure{s} - + {individual} """) joined = [] for i, (lineno, f) in enumerate(failures): - line = "Failure [{}]: {} (log line {})".format(i, f.strip(), lineno) + line = "Failure [{}]: {} (log line {})".format(i, + f.strip(), + lineno) joined.append(line) joined = '\n'.join(joined) From 30c917444c1618a8784f035f1ab383d1979a6f97 Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Thu, 20 Sep 2018 12:39:59 -0500 Subject: [PATCH 25/31] Fix call --- ci/build_docs.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ci/build_docs.sh b/ci/build_docs.sh index 84eb8dec17bae..e12d7b32544cd 100755 --- a/ci/build_docs.sh +++ b/ci/build_docs.sh @@ -29,8 +29,8 @@ if [ "$DOC" ]; then echo # Lint build log # echo ################## - echo './make.py lint_log --logfile=doc-build.log' - ./make.py lint_log --logfile=doc-build.log + echo './make.py lint_log --log-file=doc-build.log' + ./make.py lint_log --log-file=doc-build.log if [ ?$ == 1 ] then From 5e0b275d0b965b19e66fbb883a6e6d150e4858ca Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Thu, 20 Sep 2018 14:20:46 -0500 Subject: [PATCH 26/31] write a parser --- doc/make.py | 78 ++++++++++++++++++++++++++---------------- pandas/core/generic.py | 4 +++ 2 files changed, 52 insertions(+), 30 deletions(-) diff --git a/doc/make.py b/doc/make.py index 790c10285a9d5..08adf8c585ca4 100755 --- a/doc/make.py +++ b/doc/make.py @@ -14,10 +14,12 @@ import importlib import sys import os +import re import textwrap import shutil # import subprocess import argparse +from collections import namedtuple from contextlib import contextmanager import webbrowser import jinja2 @@ -328,41 +330,57 @@ def spellcheck(self): def lint_log(self): with open(self.log_file) as f: - log = f.readlines() + log = f.read() - failures = self._check_log(log) - if failures: - self._report_failures(failures) + tokens = tokenize_log(log) + failed = [tok for tok in tokens if tok.kind != 'OK'] + if failed: + report_failures(failed) sys.exit(1) - @staticmethod - def _check_log(log): - # type: (List[str]) -> List[Tuple[int, str]] - failures = [] - for i, line in enumerate(log): - if "WARNING:" in line: - failures.append((i, line)) +# ------ +# Linter +# ------ - return failures +LinterToken = namedtuple("Token", ['kind', 'value']) +IPY_ERROR = r'(?P>>>-*\n.*?<<<-*\n)' +SPHINX_WARNING = r'(?P^[^\n]*?: WARNING:.*?$\n?)' +OK = r'(?P^.*?\n)' - @staticmethod - def _report_failures(failures): - tpl = textwrap.dedent("""\ - {n} failure{s} - - {individual} - """) - joined = [] - for i, (lineno, f) in enumerate(failures): - line = "Failure [{}]: {} (log line {})".format(i, - f.strip(), - lineno) - joined.append(line) - joined = '\n'.join(joined) - - print(tpl.format(n=len(failures), - s="s" if len(failures) != 1 else "", - individual=joined)) + +def tokenize_log(log): + master_pat = re.compile("|".join([IPY_ERROR, SPHINX_WARNING, OK]), + flags=re.MULTILINE | re.DOTALL) + + def generate_tokens(pat, text): + scanner = pat.scanner(text) + for m in iter(scanner.match, None): + yield LinterToken(m.lastgroup, m.group(m.lastgroup)) + + tok = list(generate_tokens(master_pat, log)) + return tok + + +def report_failures(failed): + tpl = textwrap.dedent("""\ + {n} failure{s} + + {individual} + """) + joined = [] + for i, tok in enumerate(failed): + line = "Failure [{}]: {}".format(i, tok.value.strip()) + joined.append(line) + joined = '\n'.join(joined) + + print(tpl.format(n=len(failed), + s="s" if len(failed) != 1 else "", + individual=joined)) + + +# --- +# CLI +# --- def main(): diff --git a/pandas/core/generic.py b/pandas/core/generic.py index 4e589cb7e105f..34633d5707264 100644 --- a/pandas/core/generic.py +++ b/pandas/core/generic.py @@ -4199,6 +4199,10 @@ def head(self, n=5): on position. It is useful for quickly testing if your object has the right type of data in it. + .. ipython:: python + + 2 / 0 + Parameters ---------- n : int, default 5 From 384ace87844d66440becebb2006c1ed1b051d6ee Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Thu, 20 Sep 2018 14:26:40 -0500 Subject: [PATCH 27/31] try to exit --- ci/build_docs.sh | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/ci/build_docs.sh b/ci/build_docs.sh index e12d7b32544cd..0497cddad29be 100755 --- a/ci/build_docs.sh +++ b/ci/build_docs.sh @@ -1,4 +1,5 @@ #!/bin/bash +set -e if [ "${TRAVIS_OS_NAME}" != "linux" ]; then echo "not doing build_docs on non-linux" @@ -37,6 +38,9 @@ if [ "$DOC" ]; then echo "Errors in documentation build." exit 1 fi +fi + +if [ -z "${PANDAS_GH_TOKEN}" ]; then echo ######################## echo # Create and send docs # From 6a2a0606b2749dddbeebe8a956c108390b1bb0ca Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Thu, 20 Sep 2018 15:39:06 -0500 Subject: [PATCH 28/31] lint --- doc/make.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/doc/make.py b/doc/make.py index 08adf8c585ca4..f31ec079cf5ef 100755 --- a/doc/make.py +++ b/doc/make.py @@ -144,7 +144,7 @@ def _process_single_doc(self, single_doc): # for names not in the top-level namespace by default, # e.g. pandas.io.formats.style.Styler importlib.import_module('.'.join([obj.__name__, name])) - except ModuleNotFoundError: + except ImportError: pass obj = getattr(obj, name) except AttributeError: @@ -338,6 +338,7 @@ def lint_log(self): report_failures(failed) sys.exit(1) + # ------ # Linter # ------ From 378bd6dd7d664c64dee55ae15f2fd1198852a81b Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Thu, 20 Sep 2018 15:51:17 -0500 Subject: [PATCH 29/31] Rework doc build --- .travis.yml | 6 ++-- ci/build_docs.sh | 73 --------------------------------------- ci/docs/build_docs.sh | 23 ++++++++++++ ci/{ => docs}/doctests.sh | 0 ci/docs/lint_docs.sh | 15 ++++++++ ci/docs/upload_docs.sh | 38 ++++++++++++++++++++ ci/run_build_docs.sh | 10 ------ 7 files changed, 80 insertions(+), 85 deletions(-) delete mode 100755 ci/build_docs.sh create mode 100755 ci/docs/build_docs.sh rename ci/{ => docs}/doctests.sh (100%) create mode 100644 ci/docs/lint_docs.sh create mode 100644 ci/docs/upload_docs.sh delete mode 100755 ci/run_build_docs.sh diff --git a/.travis.yml b/.travis.yml index 40baee2c03ea0..e29e1cc0c37f5 100644 --- a/.travis.yml +++ b/.travis.yml @@ -111,17 +111,19 @@ before_script: script: - echo "script start" - - ci/run_build_docs.sh - ci/script_single.sh - ci/script_multi.sh - ci/lint.sh - ci/doctests.sh + - ci/docs/build_docs.sh + - ci/docs/lint_docs.sh - echo "checking imports" - source activate pandas && python ci/check_imports.py - echo "script done" after_success: - - ci/upload_coverage.sh + - ci/upload_coverage.sh + - ci/docs/upload_docs.sh after_script: - echo "after_script start" diff --git a/ci/build_docs.sh b/ci/build_docs.sh deleted file mode 100755 index 0497cddad29be..0000000000000 --- a/ci/build_docs.sh +++ /dev/null @@ -1,73 +0,0 @@ -#!/bin/bash -set -e - -if [ "${TRAVIS_OS_NAME}" != "linux" ]; then - echo "not doing build_docs on non-linux" - exit 0 -fi - -cd "$TRAVIS_BUILD_DIR" -echo "inside $0" - -if [ "$DOC" ]; then - - echo "Will build docs" - - source activate pandas - - mv "$TRAVIS_BUILD_DIR"/doc /tmp - mv "$TRAVIS_BUILD_DIR/LICENSE" /tmp # included in the docs. - cd /tmp/doc - - echo ############################### - echo # Log file for the doc build # - echo ############################### - - echo './make.py 2>&1 | tee doc-build.log' - ./make.py 2>&1 | tee doc-build.log - - echo ################## - echo # Lint build log # - echo ################## - - echo './make.py lint_log --log-file=doc-build.log' - ./make.py lint_log --log-file=doc-build.log - - if [ ?$ == 1 ] - then - echo "Errors in documentation build." - exit 1 - fi -fi - -if [ -z "${PANDAS_GH_TOKEN}" ]; then - - echo ######################## - echo # Create and send docs # - echo ######################## - - cd /tmp/doc/build/html - git config --global user.email "pandas-docs-bot@localhost.foo" - git config --global user.name "pandas-docs-bot" - - # create the repo - git init - - touch README - git add README - git commit -m "Initial commit" --allow-empty - git branch gh-pages - git checkout gh-pages - touch .nojekyll - git add --all . - git commit -m "Version" --allow-empty - - git remote remove origin - git remote add origin "https://${PANDAS_GH_TOKEN}@github.com/pandas-dev/pandas-docs-travis.git" - git fetch origin - git remote -v - - git push origin gh-pages -f -fi - -exit 0 diff --git a/ci/docs/build_docs.sh b/ci/docs/build_docs.sh new file mode 100755 index 0000000000000..93d0cc1455a7d --- /dev/null +++ b/ci/docs/build_docs.sh @@ -0,0 +1,23 @@ +#!/bin/bash +set -e + +cd "$TRAVIS_BUILD_DIR" +echo "inside $0" + +if [ "$DOC" ]; then + + echo "[building docs]" + + source activate pandas + + mv "$TRAVIS_BUILD_DIR"/doc /tmp + mv "$TRAVIS_BUILD_DIR/LICENSE" /tmp # included in the docs. + cd /tmp/doc + + echo './make.py 2>&1 | tee doc-build.log' + ./make.py 2>&1 | tee doc-build.log +else + echo "[skipping docs]" +fi + +exit 0 diff --git a/ci/doctests.sh b/ci/docs/doctests.sh similarity index 100% rename from ci/doctests.sh rename to ci/docs/doctests.sh diff --git a/ci/docs/lint_docs.sh b/ci/docs/lint_docs.sh new file mode 100644 index 0000000000000..c9c9b35606290 --- /dev/null +++ b/ci/docs/lint_docs.sh @@ -0,0 +1,15 @@ +#!/bin/bash +set -e + +echo "inside $0" + +if [ "$DOC" ]; then + cd /tmp/doc + + echo "[linting docs]" + + echo './make.py lint_log --log-file=doc-build.log' + ./make.py lint_log --log-file=doc-build.log +else + echo "[skipping doc lint]" +fi diff --git a/ci/docs/upload_docs.sh b/ci/docs/upload_docs.sh new file mode 100644 index 0000000000000..16b69b1e452d6 --- /dev/null +++ b/ci/docs/upload_docs.sh @@ -0,0 +1,38 @@ +#!/bin/bash +set -e + +echo "inside $0" + +if [ "${DOC}" ] && [ "${TRAVIS_PULL_REQUEST}" = "false" ]; then + + echo ######################## + echo # Create and send docs # + echo ######################## + + cd /tmp/doc/build/html + git config --global user.email "pandas-docs-bot@localhost.foo" + git config --global user.name "pandas-docs-bot" + + # create the repo + git init + + touch README + git add README + git commit -m "Initial commit" --allow-empty + git branch gh-pages + git checkout gh-pages + touch .nojekyll + git add --all . + git commit -m "Version" --allow-empty + + git remote remove origin + git remote add origin "https://${PANDAS_GH_TOKEN}@github.com/pandas-dev/pandas-docs-travis.git" + git fetch origin + git remote -v + + git push origin gh-pages -f +else + echo "[skipping doc upload]" +fi + +exit 0 diff --git a/ci/run_build_docs.sh b/ci/run_build_docs.sh deleted file mode 100755 index 2909b9619552e..0000000000000 --- a/ci/run_build_docs.sh +++ /dev/null @@ -1,10 +0,0 @@ -#!/bin/bash - -echo "inside $0" - -"$TRAVIS_BUILD_DIR"/ci/build_docs.sh 2>&1 - -# wait until subprocesses finish (build_docs.sh) -wait - -exit 0 From 8859f971e87a6904f5b9a6970e8bbd77735e3ee2 Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Thu, 20 Sep 2018 16:57:20 -0500 Subject: [PATCH 30/31] executable scripts --- .travis.yml | 2 +- ci/docs/lint_docs.sh | 0 ci/docs/upload_docs.sh | 0 3 files changed, 1 insertion(+), 1 deletion(-) mode change 100644 => 100755 ci/docs/lint_docs.sh mode change 100644 => 100755 ci/docs/upload_docs.sh diff --git a/.travis.yml b/.travis.yml index e29e1cc0c37f5..634426a70005f 100644 --- a/.travis.yml +++ b/.travis.yml @@ -114,7 +114,7 @@ script: - ci/script_single.sh - ci/script_multi.sh - ci/lint.sh - - ci/doctests.sh + - ci/docs/doctests.sh - ci/docs/build_docs.sh - ci/docs/lint_docs.sh - echo "checking imports" diff --git a/ci/docs/lint_docs.sh b/ci/docs/lint_docs.sh old mode 100644 new mode 100755 diff --git a/ci/docs/upload_docs.sh b/ci/docs/upload_docs.sh old mode 100644 new mode 100755 From b9331f19cc9e6842a5942154291ae6b01fc761fe Mon Sep 17 00:00:00 2001 From: Tom Augspurger Date: Thu, 20 Sep 2018 17:47:07 -0500 Subject: [PATCH 31/31] activate --- .travis.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.travis.yml b/.travis.yml index 634426a70005f..502f06fbd329a 100644 --- a/.travis.yml +++ b/.travis.yml @@ -122,6 +122,7 @@ script: - echo "script done" after_success: + - source activate pandas - ci/upload_coverage.sh - ci/docs/upload_docs.sh