Mailing List Archive

convert Dbase (.dbf) files to SQLite databases
Hi,

I have a lot of old Dbase files (.dbf) and I'll like to convert these
to SQLite databases as automatically as possible.
Does anybody know a tool/Python script to do so?

I know, I could use dbfpy and create the SQLite table and import all
data. But is there something easier?

Many thanks for a hint,

Helmut.

--
Helmut Jarausch

Lehrstuhl fuer Numerische Mathematik
RWTH - Aachen University
D 52056 Aachen, Germany
--
http://mail.python.org/mailman/listinfo/python-list
Re: convert Dbase (.dbf) files to SQLite databases [ In reply to ]
On Wed, 15 Jul 2009 11:53:28 +0200, Helmut Jarausch
<jarausch@igpm.rwth-aachen.de> wrote:
> Hi,
>
> I have a lot of old Dbase files (.dbf) and I'll like to convert these
> to SQLite databases as automatically as possible.
> Does anybody know a tool/Python script to do so?
>
> I know, I could use dbfpy and create the SQLite table and import all
> data. But is there something easier?

yes...

Use OpenOffice-Scalc or MS-Office-Excel to open the table...

Export to csv....

Use SQLite Manager (https://addons.mozilla.org/en-US/firefox/addon/5817)

and use the import wizard to import your data....

It shouldn't take too long...

David
--
http://mail.python.org/mailman/listinfo/python-list
Re: convert Dbase (.dbf) files to SQLite databases [ In reply to ]
Helmut Jarausch wrote:
> Hi,
>
> I have a lot of old Dbase files (.dbf) and I'll like to convert these
> to SQLite databases as automatically as possible.
> Does anybody know a tool/Python script to do so?
>
> I know, I could use dbfpy and create the SQLite table and import all
> data. But is there something easier?
>
> Many thanks for a hint,
>
> Helmut.
>

Greetings!

If your tables are either dBase III or Visual Foxpro 6 (may work with
other VFP versions...) you can try
http://groups.google.com/group/python-dbase

#open table
import dbf
table = dbf.Table('/path/to/old/dbf')

#generate .csv file
table.export() # defaults to same name with csv extension

#access records
for rec in table:
tups = tuple(rec)
lst = list(rec)
dct = rec.scatter_fields()

I haven't used SQlite, so I'm unable to provide samples for that portion
of the conversion.

Hope this helps.

~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list
Re: convert Dbase (.dbf) files to SQLite databases [ In reply to ]
On Jul 15, 8:39 pm, David Lyon <david.l...@preisshare.net> wrote:
> On Wed, 15 Jul 2009 11:53:28 +0200, Helmut Jarausch
>
> <jarau...@igpm.rwth-aachen.de> wrote:
> > Hi,
>
> > I have a lot of old Dbase files (.dbf) and I'll like to convert these
> > to SQLite databases as automatically as possible.
> > Does anybody know a tool/Python script to do so?
>
> > I know, I could use dbfpy and create the SQLite table and import all
> > data. But is there something easier?
>
> yes...
>
> Use OpenOffice-Scalc or MS-Office-Excel to open the table...

Max 64K rows for Scalc and Excel 2003; 2007 can take 2**20 rows.
Only old dBase (not dBase 7). Memo fields not handled. Visual FoxPro
DBFs not supported by Excel even tho' VFP is an MS product.


> Export to csv....

Yuk.

>
> Use SQLite Manager (https://addons.mozilla.org/en-US/firefox/addon/5817)
>
> and use the import wizard to import your data....
>
> It shouldn't take too long...

... before you get sick of the error-prone manual tasks.

I'd write a script that took a DBF file, analysed the field
descriptions, made a CREATE TABLE statement, executed it, and then
started doing inserts. Fairly easy to write. Scripts have the great
benefit that you can fix them and re-run a whole lot easier than
redoing manual steps.

If dbfpy can't handle any new-fangled stuff you may have in your
files, drop me a line ... I have a soon-to-be released DBF module that
should be able to read the "new" stuff up to dBase7 and VFP9,
including memo files, conversion from whatever to Unicode if
needed, ...

Cheers,
John
--
http://mail.python.org/mailman/listinfo/python-list
Re: convert Dbase (.dbf) files to SQLite databases [ In reply to ]
John Machin wrote:

> If dbfpy can't handle any new-fangled stuff you may have in your
> files, drop me a line ... I have a soon-to-be released DBF module that
> should be able to read the "new" stuff up to dBase7 and VFP9,
> including memo files, conversion from whatever to Unicode if
> needed, ...
>
> Cheers,
> John

Cool! I'm looking forward to it!

~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list
Re: convert Dbase (.dbf) files to SQLite databases [ In reply to ]
Il giorno mercoledì 15 luglio 2009 18:30:29 UTC+2, John Machin ha scritto:
> On Jul 15, 8:39 pm, David Lyon <david.l...@preisshare.net> wrote:
> > On Wed, 15 Jul 2009 11:53:28 +0200, Helmut Jarausch
> >
> > <jarau...@igpm.rwth-aachen.de> wrote:
> > > Hi,
> >
> > > I have a lot of old Dbase files (.dbf) and I'll like to convert these
> > > to SQLite databases as automatically as possible.
> > > Does anybody know a tool/Python script to do so?
> >
> > > I know, I could use dbfpy and create the SQLite table and import all
> > > data. But is there something easier?
> >
> > yes...
> >
> > Use OpenOffice-Scalc or MS-Office-Excel to open the table...
>
> Max 64K rows for Scalc and Excel 2003; 2007 can take 2**20 rows.
> Only old dBase (not dBase 7). Memo fields not handled. Visual FoxPro
> DBFs not supported by Excel even tho' VFP is an MS product.
>
>
> > Export to csv....
>
> Yuk.
>
> >
> > Use SQLite Manager (https://addons.mozilla.org/en-US/firefox/addon/5817)
> >
> > and use the import wizard to import your data....
> >
> > It shouldn't take too long...
>
> ... before you get sick of the error-prone manual tasks.
>
> I'd write a script that took a DBF file, analysed the field
> descriptions, made a CREATE TABLE statement, executed it, and then
> started doing inserts. Fairly easy to write. Scripts have the great
> benefit that you can fix them and re-run a whole lot easier than
> redoing manual steps.
>
> If dbfpy can't handle any new-fangled stuff you may have in your
> files, drop me a line ... I have a soon-to-be released DBF module that
> should be able to read the "new" stuff up to dBase7 and VFP9,
> including memo files, conversion from whatever to Unicode if
> needed, ...
>
> Cheers,
> John

--
https://mail.python.org/mailman/listinfo/python-list
Re: convert Dbase (.dbf) files to SQLite databases [ In reply to ]
> Il giorno mercoledì 15 luglio 2009 18:30:29 UTC+2, John Machin ha scritto:

>> If dbfpy can't handle any new-fangled stuff you may have in your
>> files, drop me a line ... I have a soon-to-be released DBF module that
>> should be able to read the "new" stuff up to dBase7 and VFP9,
>> including memo files, conversion from whatever to Unicode if
>> needed, ...

Or you can use my dbf module [1] which has been in the wild for several
years now and is fairly robust instead of waiting for John's module
which has been in the "soon to be released" category for nearly as long.

--
~Ethan~

[1] https://pypi.python.org/pypi/dbf
--
https://mail.python.org/mailman/listinfo/python-list
Re: convert Dbase (.dbf) files to SQLite databases [ In reply to ]
>> Il giorno mercoledì 15 luglio 2009 18:30:29 UTC+2, John Machin ha
>> scritto:

Hmm, looks like that email was sent back in 2009. Sorry for the noise.
I'm still not able to find a dbf module from John, though.

--
~Ethan~
--
https://mail.python.org/mailman/listinfo/python-list
Re: convert Dbase (.dbf) files to SQLite databases [ In reply to ]
On 03/01/2016 06:08 PM, Dennis Lee Bieber wrote:
> On Tue, 01 Mar 2016 08:49:08 -0800, Ethan Furman declaimed the following:

>> Hmm, looks like that email was sent back in 2009. Sorry for the noise.
>> I'm still not able to find a dbf module from John, though.
>>
> Is it one of:
>
> https://www.google.com/#q=dbf.py

I only checked the first three pages:

- 12 refer to mine
- 4 refer to pydbf
- various others

The third page drifts off into non-English languages and non-Python
links. A search for his name and dbf also did not reveal anything.

Ah well (aka *sigh*).

--
~Ethan~
--
https://mail.python.org/mailman/listinfo/python-list