Exporting ISO date in Numpy

I am new in python and have a basic question I can not find a solution on the net.

I have a table named IBTsample.txt, containing:

  • Season,ISO_time,Latitude,Longitude,Enum
  • 2009,2009-12-24 12:00:00,6.50,85.00,2
  • 2009,2009-12-25 06:00:00,8.00,84.50,6
  • 2009,2009-12-25 00:00:00,7.00,84.50,4
  • 2009,2009-12-24 18:00:00,6.50,85.00,3
  • 2009,2009-12-24 09:00:00,6.50,85.00,1
  • 2009,2009-12-25 03:00:00,7.00,84.50,5

What I would like to do is import it as a numpy array, do some processing (for the time being simply sort the records par date) and export the processed table in a new .txt file.

from numpy import * import pylab rawtable = loadtxt('IBTsample.txt', delimiter=',', skiprows=1, converters= {1:pylab.datestr2num},\ dtype={'names':('Season','ISO_time','Latitude','Longitude','Enum'),\ 'formats':('uint16','float','float16','float16','uint8')}) sortable = (sort(rawtable, order=('ISO_time'))).copy() savetxt('IBTsorted.txt', sortable, fmt='%d,%.3f,%.1f,%.1f,%d')

I have ti use the pylab.datestr2num function during the importation but can not find an inverse function to export the date and time in an ISO format.

Any help or idea would be helful, thanks in advance.

-------------Problems Reply------------

You could use matplotlib.dates.num2date to convert the nums back into datetime objects. Then call isoformat() to get the date as a string in ISO-8601 format.

import numpy as np
import matplotlib.dates as md

def num2isodate(num):
result=md.num2date(num).isoformat()
return result

rawtable = np.loadtxt(
'IBTsample.txt', delimiter=',', skiprows=1,
converters= {1:md.datestr2num},
dtype={'names':['Season','ISO_time','Latitude','Longitude','Enum'],
'formats':['uint16','float','f4','f4','uint8']})

Convert the ISO_time dtype to object. This allows the column to hold float values at first, and strings later. Note that astype (below) returns a copy, so calling copy explicitly is no longer needed. Also, since you called copy, I assume holding two copies of the array in memory is not a problem. (If memory were tight, we could write the array line-by-line using the csv module instead of using np.savetxt. But since memory is not a problem, np.savetxt is more convenient.)

sortable = rawtable.astype({'names':['Season','ISO_time','Latitude','Longitude','Enum'],
'formats':['uint16','object','f4','f4','uint8']})
sortable = np.sort(sortable, order=('ISO_time'))
sortable['ISO_time'] = [num2isodate(num) for num in sortable['ISO_time']]
np.savetxt('IBTsorted.txt', sortable, fmt='%d,%s,%.1f,%.1f,%d')


PS. I'd recommend never using from module import *, especially when module is numpy. It overwrites a number of Python builtin functions, such as abs, all, any, min, max, sum, round, etc. Not only does that make it harder to call the Python builtins, but also makes it easy to write code which looks right but contains hard-to-find or subtle bugs.

In addition to @unutbu's answer, if you're using the latest version of numpy (>= 1.7), there is a native date dtype.

In that case, try something like this:

import numpy as np
import dateutil.parser

rawtable = np.genfromtxt('test_in.txt', names=True, delimiter=',',
converters={1:dateutil.parser.parse},
dtype=[np.uint16, np.datetime64, np.float, np.float, np.uint8])

sorttable = np.sort(rawtable, order=('ISO_time'))

with open('test_out.txt', 'w') as outfile:
outfile.write(','.join(sorttable.dtype.names) + '\n')
np.savetxt(outfile, sorttable, fmt='%i,%r,%f,%f,%i')

This yields:

Season,ISO_time,Latitude,Longitude,Enum
2009,2009-12-24 09:00:00,6.500000,85.000000,1
2009,2009-12-24 12:00:00,6.500000,85.000000,2
2009,2009-12-24 18:00:00,6.500000,85.000000,3
2009,2009-12-25 00:00:00,7.000000,84.500000,4
2009,2009-12-25 03:00:00,7.000000,84.500000,5
2009,2009-12-25 06:00:00,8.000000,84.500000,6

Category:python Views:0 Time:2011-11-02

Related post

  • What is the simplest way to export Excel data to Matlab? 2008-10-13

    What is the simplest way to programmatically export Excel data to Matlab? --------------Solutions------------- If you are running Matlab on Windows with Excel installed, try XLSREAD. If there's not much text annotation in the Excel file, save it as a

  • How best to export native data to Excel without introducing dependency on Office? 2008-11-10

    Our product has the requirement of exporting its native format (essentially an XML file) to Excel for viewing/editing. However, what this entails is having a dependency on Excel (or Office) itself for our product build - something that we do not want

  • How to Export SSIS data on SQL Standard and Import with Bulk Insert on SQL Express? 2009-04-09

    I would like to use SSIS to create database table flat file exports then import them in to variously distributed SQL Express installs. As you may guess, SSIS is not available in the Express version. When I do the bulk insert, it errors with: Msg 4866

  • Can any one tell me how to write a simple C++ code to export my data(from variable) into PDF file without using any external libraries or utilities? 2009-08-05

    Can any one tell me how to write a simple C++ code to export my data(from variable) into PDF file without using any external libraries or utilities? --------------Solutions------------- Use fprintf and write to a file, conforming to the PDF file form

  • Source of ISO data in xml format? 2009-09-20

    I was just wondering if anyone knew of a source of standard ISO data in xml format, such as ISO 3166 (Country codes/names), ISO 4217 (Currency Codes) or ISO 639 (Languages)? If it was in any way 'official' and maintained with any changes to the stand

  • How to map an XML schema to an OpenCalc spreadsheet cell and export cell data as an XML data file 2009-10-01

    I have created an XML schema for storing data. I want to provide a simple user interface with OpenCalc spreadsheet for data entry. Things which I wanted to do. Map my XML schema elements to cells of a spreadsheet. Provide this spreadsheet as an user

  • export canvas data to json 2009-10-15

    Hay guys, i've made a simple drawing application using the canvas tag. However i would like to export the data to JSON so it can be saved. How does one go about this? --------------Solutions------------- What you need is the toDataUrl(type) method. I

  • How can i export table data with column name in text file in sql server 2005? 2009-10-24

    How can i export table data with column name in text file in sql server 2005? --------------Solutions------------- safest way to do that is to backup data and then restore it on other server! right click on database in ms sql management studio>tas

  • Jasper report exports empty data in PDF format when there is more data 2009-10-29

    I have a report to be exported in excel, pdf and word using jasper reports. I use xml file as the DataSource for the report, but when the data increases jasper report exports empty file in only for PDF format, when i reduce the data content it export

  • Export the data into a PDF format with Tomahawk? 2010-02-23

    In my application I am trying to export the data from the <h:DataTable> into a PDF file using Apache TomaHawk. While loading the page I am getting an exception. I have Pasted my code and the exception below. JSF Code <%@ page session="false"

  • Export view data programmatically in Access/SQL Server 2010-04-26

    We have an Access application front-end connected to a SQL Server 2000 database. We would like to be able to programmatically export the results of some views to whatever format we can (ideally Excel, but CSV / tab delimited is fine). Up until now we

  • Ruby (Rails): export db data for SPSS 2010-05-21

    I have a Rails application to do web surveys. It stores answers to multiple-choices (fixnum) questions into sqlite3. Is there any ruby gem that I can use to export my data into spss format ? I'd like to be able to export for SPSS version prior of 16/

  • How to export some data into .rtf or .doc? 2010-06-12

    Say I have created a GUI (Qt) that operates with some data(text+image). How I can export that data in one of listed formates in title. I guess it is better to export info .rtf as it is free cross-platform formt(??), isn't it? --------------Solutions-

  • Import/Export XML data into SQL Server Compact Edition? 2010-07-08

    Does it exist a Microsoft tool or a sql query for import/export XML data into SQL Server Compact Edition ? Like this for SQL Server : http://msdn.microsoft.com/en-us/library/ms191184.aspx#binary_byte_stream --------------Solutions------------- I have

  • Export a data set to Excel and raise a file download dialog from an asp.net web method 2010-08-06

    I am using the following code to export a data set to an Excel sheet. [WebMethod] public static void ExporttoExcel() { DataSet ds; productfactory pf=new productfactory(); ds = pf.getproducts(); HttpResponse response = HttpContext.Current.Response; //

  • is it possible to export XML data from a swf file 2010-09-02

    is it possible to export XML data from a swf file --------------Solutions------------- http://rickydoesit.com/actionscript-3/how-to-connect-to-php-with-actionscript-3-and-post-variables-to-php/ http://www.adobe.com/devnet/flash/articles/mysql_xml_asp

  • export oracle data block to excel 2010-11-11

    according to this post I try to export my data to Excel, after several challenge now my problem is it seems work well but I can not see the exported excel file!!!! (first solution's http://www.protalk.in/oracle/plsql-tips-package-for-exporting-data-t

  • Can we export graphical data to pdf using php scripts? 2010-12-03

    Can we export graphical data to pdf using php scripts? Suppose the graphs are rendered by flash and html css , can we export these graphical data to a pdf. Any pointers would be greatly appreciated... --------------Solutions------------- Read documen

  • cannot open connection while exporting the data to database using hibernate 2010-12-15

    I have written a service to export jcr data to database. The service starts itself when I start the server of my application previously It was exporting the data with no any problem since two days whenever I start the server it get successfull for ex

Copyright (C) dskims.com, All Rights Reserved.

processed in 0.104 (s). 11 q(s)